Are you really saying that judging someone that way isn't a form of antisemitism?

Not necessarily. Let me ask you this: Imagine your hypothetical left-winger who won't vote for a Rothschild. Do you think that person would vote for a "Rockefeller"? My guess is he probably wouldn't, but even if he would, he would probably invent some rationalization for it so he could pretend to himself and his peers that he is not an anti-Semite.

By the way, I do agree that much of the time, criticism of "Bankers" or "Wall Street Bankers" or "Elites who Control the Media" etc. is tinged with anti-Semitism, even when it comes from the Left.