No person, firm or corporation shall commence any excavation, grading, earth filling or earth removal or commence any erection, construction, enlargement, alteration, removal, improvement, demolition, conversion or change in the nature of the use or occupancy of any building or structure or cause the same to be done without first obtaining a separate zoning permit from the Zoning Officer of the Town of Brutus for each such lot, such building or structure, except that no zoning permit shall be required for the performance of ordinary repairs which are not structural in nature.