The site shall be at least 20 acres for a tennis, skating or swimming club and at least 60 acres for a golf or riding club. A special use permit may be granted only when the petitioner files with the Board of Appeals written consents, duly acknowledged, of the owners of at least 75% of all land which lies outside of and within 1,000 feet of each boundary line of the lot or lots of the petitioner to be used for such conditional use, excluding however, in such one-thousand-foot computation, all the land contained in public streets, and all the land owned by petitioner which is adjacent to the lot or lots intended to be used for such conditional use, and, if the petitioner shall be a prospective purchaser with a contract of purchase, all lands which lie adjacent to the lot or lots to be used for such conditional use owned by the seller; provided, however, that notwithstanding such limitation, the Board of Appeals, by majority vote of all the members of the Board then in office, may grant permission for any such use even though consents are not so filed on behalf of the owners of at least 75% of the above-mentioned area.