If a building permit is denied or the application is found to require site plan review, the Building Official shall promptly cause to be sent to the applicant, addressed to the address of the applicant set forth in the application for the said permit, a notice of his/her action, which shall specify, in writing, the ground or grounds upon which the same is based. In the case of a denial, applicant may, within 20 days thereafter, notify the Building Official in writing that he/she appeals from such action of the Building Official to the Board of Adjustment, and the Building Official shall then immediately transmit to the Board of Adjustment all the papers constituting the record upon which the action appealed from is taken, and the Board of Adjustment shall, within 120 days thereafter and upon five days' notice in writing to the said owner or his/her agent, review the determination of the Building Official and either affirm the determination of the Building Official or reverse same or issue a variance from the provisions of this chapter.