No building or structure shall be erected, added to or structurally altered until a permit has been issued by the Building Inspector. Except upon a written order of the Board of Adjustment, no such building permit shall be issued for any building where said construction, addition, alteration or use thereof would be in violation of any of the provisions of this Chapter.