After a nonconforming building or use has been converted to a permitted use it shall not be changed back again to a nonconforming use.