Nothing in this chapter shall prevent the restoration of a nonconforming building partially destroyed by fire, explosion, act of God or act of public enemy or prevent the continuance of the use of such building, or part thereof, as such use existed at the time of such partial destruction of such building, or part thereof, or prevent a change of such existing use under the limitations provided herein.