The pool structure, including perimeter walkway, paved areas or accessory structures, except the required fence, shall not be located closer than 15 feet from any side or rear property line nor closer to any street than the minimum building setback line permitted for the district in which it is located.