The Board of Adjustment may authorize a variance to permit the conversion of any building into a two-family or multiple-family dwelling in any permissible residential district, provided that the lot area per family shall not be reduced to less than the lot area required for such conversion in each of the respective residential districts.