Any sign previously given approval by the Board which otherwise does not conform to this section shall be deemed a permitted sign, unless the use for which such sign had been permitted shall have been abandoned.