Any building permit heretofore issued pursuant to any ordinance which provided for the erection of a building or structure in conformance to the ordinance or ordinances in effect prior to the date of adoption of this chapter shall continue in full force and effect, provided that the holder has substantially undertaken such construction on the date of adoption of this ordinance.