On appeal from an order, requirement, decision or determination made by the Building Inspector or on referral of an applicant to the Board by an approving agency acting pursuant to this chapter, the Zoning Board of Appeals is authorized to vary or modify the strict letter of this chapter where its literal interpretation would cause practical difficulties or unnecessary hardships, as defined in this section, in such manner as to observe the spirit of the chapter, to secure public safety and welfare and to do substantial justice.