Single-family and two-family residential structures and structures and uses incidental thereto, permitted as a right under applicable zoning regulations, are exempt from the requirements of this Sections 35-18 through 35-23; however, a site plan will be required if, at the discretion of the Building Officer, there exist soil and drainage conditions or traffic factors which may result in environmental problems.