The accessory apartment shall not adversely affect the character of the neighborhood in which it is located. In applying this requirement, the Planning Board shall consider the effect of the proposed accessory apartment on parking, traffic, noise, congestion, appearance and any other factor that the Planning Board deems relevant to the character of the neighborhood. The Planning Board may refuse to issue a special permit if it finds that the number of such approved apartments in the neighborhood, including the one proposed, will adversely affect the character of the neighborhood.