Any structure, building, sign or other improvement lawfully existing at the time of adoption or amendment of this chapter but which is nonconforming by dimension shall be permitted to continue in the same manner. If said legally nonconforming structure, building, sign or other improvement is intentionally destroyed, relocated or altered, it shall lose its nonconforming status. However, if such destruction, relocation or alteration is the result of fire or a natural catastrophe, the legal nonconforming improvement may be rebuilt in its previous configuration.