Except as otherwise provided herein, the lawful use of land or buildings, existing at the effective date of adoption of this chapter, may be continued although such use does not conform to the provisions hereof. If no structural alterations are made, a nonconforming use of a building may be changed to another nonconforming use of the same or of a more restricted classification. Whenever a nonconforming use of land or buildings has been changed to a more restricted use or to a conforming use, such use shall not thereafter be changed to a less restricted use. The nonconforming use of a building may be hereafter extended throughout those parts of a building which were lawfully and manifestly arranged or designed for such use at the time of the adoption of this chapter.