In 1921 the Memorial Building was voted to be built in memory of Ipswich Veterans of all wars at a cost of $52,000. It was used to house two 6th grade classes (52 pupils) from 1969-1971 and as an office facility for the Pupil Personnel department during the years 1974-1980.
It was also at different times used by many committees and organizations for different purposes including as an office by both Jim Daley and Betty Dorman while they were Recreation Directors, and also for many Teen club meetings and events.
The long stairs to reach the building from Central Street became problematic as towns needed to make their facilities handicapped accessible. The stairs were removed and a stone wall was erected along Central Street. The town sold the building to Memorial Hall Housing Inc., a non-profit corporation in 2003. HOME funds were used to partially fund the renovation of Memorial Hall into affordable elderly housing.