This chapter is not intended to abrogate or annul any building permit, certificate of occupancy, variance or special permit lawfully issued before the effective date of this chapter, except as follows: If the applicable regulations of this chapter or any amendments to the chapter after issuance of a building permit, granting of a variance or issuance of a special permit make the proposed use under such building permit, variance or special permit nonconforming as to the Schedule of Uses or Schedule of Bulk Regulations, and no substantial construction or substantial operations for nonbuilding uses have been undertaken on the structure or foundation or conducted for the nonbuilding uses within 120 days after the effective date of the adoption of this chapter and any amendments thereto, the building permit, variance or special permit shall be invalid. If substantial construction or substantial operations have taken place and are continuing at the time, the proposed use may be completed.