When a nonconforming use has been discontinued for a period of one year or more, such use shall be deemed abandoned, with the right, however, to the owner or lessee to give proof before the Superintendent of Buildings that there was no intention to abandon such nonconforming use and to appeal from his decision to the Board of Appeals, except that where there is a nonuser of the nonconforming use for a period of two years, then the right to put said building and premises to such use shall cease and come to an end, and the premises shall thereafter be put to conforming uses only.