If the design, location, orientation or construction of any building, structure or use is so markedly incongruous with the character of the neighborhood as to materially affect the value of adjacent or nearby properties, the Construction Official or Zoning Officer, as applicable, shall deny a permit and refer the applicant to the Board of Adjustment on properties involving single-family detached dwellings or to the Planning Board, on all other properties, and such Township agency shall act thereon after an inspection of the locale, examination of the plans, and a hearing.