The Shulhan Arukh, Hoshen Mishpat 3:2 writes, “A scholar is prohibited from sitting down to a court case until he knows with whom he is sitting lest he sit with men who are unfit and becomes part of a rebellious group rather than a court.” While this halakhah only technically applies to a religious court, it is important advice for life. Be careful with whom you affiliate because you may unwittingly advance your colleagues’ causes and become perceived by the public as associated with those colleagues.
And now, a story. Over ten years ago, R. Hershel Schachter was being driven to one of his weekly lectures in Queens (I think, or maybe Brooklyn) and, on the highway, the small group was car-jacked. Yes, the car was stolen at gunpoint and they were left stranded on the highway. Over the Shabbos of that week was scheduled an NCSY event at which R. Schachter was scheduled to speak. R. Schachter apologetically backed out of his speaking engagement for that Shabbos, implying that his wife was still shaken up by the car-jacking. A friend of mine, now a pulpit rabbi and a kashrus professional, was invited to eat Shabbos dinner at the Schachter home that week and heard a very different story.
Also invited to speak at the NCSY event was a prominent Modern Orthodox rabbi and speaker whose speeches and published writings had been straddling the line of Orthodoxy, perhaps crossing it on occasion. When R. Mordechai Willig, who spoke at the NCSY event, discovered in advance that this rabbi would also be speaking, he asked R. Schachter whether he should back out of his speaking obligation. Presumably, he did not want to be associated with this rabbi. R. Schachter told R. Willig that he was not obligated to back out of his speaking obligation. However, since R. Schachter had another excuse he chose to back out of the event.[Note, I heard this second-hand so do not take it as gospel.]
The point: Be very aware of those upon whom you bestow respectability by appearing as one of their colleagues. Furthermore, the more you associate with such people, the more the public will associate you with them and their causes – justly or not.