Without a public hearing. Preexisting single- or two-family residential structures may be reconstructed, changed, extended or altered without the necessity of a public hearing, provided that it is determined that no such reconstruction, change, extension or alteration shall increase the nonconforming nature of said structure. Such determination shall be made by the Zoning Board of Appeals or the Zoning Administrator, if one has been appointed. In the event that such reconstruction, change, extension or alteration increases the nonconformity of said structure, no such reconstruction, change, extension or alteration shall be permitted unless there is a finding by the Zoning Board of Appeals pursuant to a public hearing that such reconstruction, change, extension or alteration shall not be substantially more detrimental than the existing nonconformity to the neighborhood.