No existing non-agricultural building or structure devoted to a use not permitted in the district in which such building or premises is located, except when required to do so by law, should not be enlarged, extended, reconstructed or structurally altered, unless such use is changed to a use permitted in the district in which such building or structure is located, provided however, any building containing a residential non-conforming use may be altered in any way to improve livability but no structural alteration should be made which would increase the number of dwelling units therein or the bulk of the building. Any existing building or structure devoted to a use permitted in the district in which such building or structure is situated, but failing to comply with the yard or building setback line requirements of this Chapter, may be enlarged, extended, reconstructed or structurally altered in any manner which does not further encroach upon the required open yard space or building setback line which is not complied with.