If the proposed excavation, construction, alteration or moving as set forth in the application is in conformity with the provisions of this chapter and other regulations of the Town of Caledonia then in force, the Zoning Enforcement Officer shall issue a permit for such excavation, construction, alteration or moving.