Except as may be specifically authorized in this chapter, no land shall be used and no building, structure or sign shall be erected, modified, enlarged or used unless such action conforms in all respects to all provisions of this chapter. Every building, structure or sign hereafter erected, and every use of land hereafter initiated, shall be located on a lot or site as defined by this chapter, and there shall be no more than one main or principal residential building, together with its permitted accessory structures, on a lot, except where specifically authorized by this chapter. The Town of Barrington itself, both as to land owned by the Town and to governmental activity and use, shall be exempt from the provisions of this chapter.