In all districts, required garages or parking areas accessory to any permitted nonresidential use shall be on the same lot or parcel of land with the main building, buildings or use, except that where there are practical difficulties or unnecessary hardships, the Board of Appeals may grant a permit for a required parking area to be located on other property, subject to appropriate safeguards, within 500 feet of the property to which it is accessory, measured in a straight line between the nearest point of the property to be served. Such permit shall be subject to safeguards and conditions designed to assure the continuing availability of such parking area to serve the use to which it is accessory for a minimum term of 20 years; provided, however, that a different parking area may be substituted therefor at any time within such twenty-year period, on approval of the Board of Appeals and subject to similar safeguards and conditions.