A dwelling or lodging house shall not exceed 2 1/2 stories or 35 feet. Churches, schools, colleges, libraries, Town buildings, clubs, lodges, social center buildings, hospitals and such institutional buildings shall not exceed 50 feet in height. Chimneys, steeples and flag or radio poles are exempt from the height provisions.