Nothing herein contained shall require any change in plans, construction or designated use of a building, structure or sign for which a building permit has heretofore been issued and the construction of which shall have been diligently prosecuted within three months of the date of such permit, and the ground-story framework of which, including the second tier of beams, shall have been completed within six months of the date of the permit, and which entire building shall be completed according to such plan as filed within one year from the date of the adoption of this chapter.