There shall be no more than one principal structure on each lot in any residence district, except that any farm structures shall not be considered a second principal structure.