No member or alternative member of the Planning Board or Zoning Board of Adjustment shall act on any matter in which he has, either directly or indirectly, any personal or financial interest. Whenever any such member shall disqualify himself from acting on a particular matter, he shall not continue to sit with the Board on the hearing of such matter nor participate in any discussion or decision relating thereto. A member may, after public hearing if he requests it, be removed by the governing body for cause.