Except as hereinafter provided, no building or structure or part thereof shall hereafter be erected, structurally altered, enlarged or rebuilt except in conformity with the Schedule of District Regulations as to height, lot dimension, coverage, yard, court and density regulations herein prescribed. Any building or structure that does not conform to such regulations, herein referred to as the "building regulations" of this chapter, shall be deemed a nonconforming structure, irrespective of the use to which it is put. Variances granted by the Zoning Board of Appeals pursuant to §
57-65 on grounds of practical difficulties or unnecessary hardship, not self-imposed, shall be deemed nonconforming structures.