Abandonment of a nonconforming use: A nonconforming use which has been abandoned or discontinued for a period of more than two years or has been replaced by a conforming use shall lose the protection set forth above in Section 5.1, except to the extent that the nonconforming use or structure may be authorized by a special permit issued by the Zoning Board of Appeals which will be granted only if the Board finds that the proposed alteration, reconstruction or extension will not be substantially more detrimental to the neighborhood than is the existing nonconforming use, and the real estate taxes on the subject property have been continuously paid and the owner is otherwise in compliance with all the municipal ordinances, rules and regulations, including all Board of Health regulations.