After a nonconforming building or structure or use has been converted to a permitted use it shall not be changed back again to a nonconforming use.