Any use for the establishment of which a permit from the Board of Appeals is required under the use regulations for a particular district, whether such use is existing in such district at the time of the adoption of this section or is subsequently established therein on the obtaining of such a permit, shall be deemed to be a conforming use in such district, provided that any subsequent replacement or enlargement of such use or of any building in which the same is conducted or the construction of an additional building for such use, beyond the extent specified in any such permit, shall be subject to the obtaining of a further permit therefor from the Board of Appeals.