The removal of soil, loam, sand, gravel, quarry stone or other earth material from any parcel of land by any person, firm or corporation shall be allowed only under permit from the Board of Zoning Appeals and subject to restrictive covenants of the Board. However, the Inspector of Buildings, without reference to the Board of Zoning Appeals, may issue permits for the removal from the site of earth and may require a suitable bond, if deemed necessary, for the following purposes: wherein the amount of material to be removed is necessarily incidental to or in connection with the construction, alteration, excavation or grading for a building or road or other facility being built in accordance with a permit issued or in accordance with an approved plan, provided that there is reasonable assurance that the construction will be completed.