If a nonconforming structure or building, or portions thereof, containing a nonconforming use becomes physically unsafe or unlawful due to lack of repairs or maintenance, and is declared by any duly authorized official to be unsafe or unlawful by reason of physical condition, such structure or building shall not thereafter be restored, repaired, or rebuilt except in conformity with the regulations of the zoning district in which it is located.