Nothing herein contained shall require any change in the overall layout, plans, construction size or designated use of any building, or part thereof, for which approvals and required building permits have been granted before the enactment of these regulations. The construction in conformance with such plans shall have been started prior to the effective date of these regulations and completion thereof carried on in a normal manner and not discontinued for reasons other than those beyond the builder’s control. Construction shall be considered to have started when the first construction materials have been placed in permanent position and fastened in a permanent manner. Demolition or removal of an existing building begun preparatory to rebuilding shall be deemed to be actual construction, provided that such work shall be diligently carried on until completion of the building involved.