Only that portion of the land or water in actual use may be so continued, and the use may not be extended, enlarged, substituted or moved, and the structure may not be extended, enlarged, reconstructed, substituted, moved, or structurally altered, except when required to do so by law or order or so as to comply with the provisions of this chapter.