Building zone districts include all lots, streets, ponds, streams, waterways and bays therein or adjacent thereto, whether or not so designated upon the Zoning Map, including lands under water and lands formed by filling, accretion, emergency avulsion or otherwise. In case a street, pond, stream or waterway, as shown on said map, forms a boundary between two or more districts, each district shall be deemed to extend to the middle line of the street or the center of the pond or the thread of the stream or waterway, as the case may be.