When a non-conforming building, or a building in which a non-conforming use is conducted, is destroyed or damaged by any casualty to an extent not exceeding 50% of its true value, exclusive of foundations, at the time of such destruction or damage, it may be reconstructed but it shall not be enlarged. If any such building is so destroyed or damaged to an extent exceeding 50% of its true value, it shall not be reconstructed except for a conforming use; provided, that if any non-conforming building used as a one-family dwelling or two family dwelling is so destroyed or damaged to an extent exceeding 50% of its true value, the same may be reconstructed within two years after such destruction or damage so as to be the same space that is occupied prior to such destruction or damage or any part thereof.