What are the bad things that can happen to you if you get accurately judged as nerdy

This brings up an excellent point.

It's perfectly fine to be Packers fan, but I would judge a person who showed up to a wedding or funeral dressed as a cheesehead. I wouldn't judge them for being a Packards fan; I would judge them for disrespectfully violating decorum -- for choosing to signal that they are a Packards fan.

Projecting nerdiness is similar EDIT in that the first-order signal is not the harmful one. A nerdy appearance emits two important signals:

1. You are nerdy
2. You are a person who emits signals that they are nerdy

The second-order signal is the more negative and important. It isn't too bad if people think you are nerdy; it's bad if people think that you don't understand or don't care how most people perceive you. It signals a lack of self-awareness, or a deficient understanding of cultural norms, or blithe indifference. In my case, it accurately signaled all three.

Signaling my lack of self-awareness didn't cause bad things to happen; it prevented good things from happening.

how do they compare to the negative impact of having people assume all the wrong things about you?

No negative impact, and I don't think people do assume the wrong things about me. In conversation I'm usually transparent about who I am. You can credibly profess to be nerdy no matter what clothing you wear. You cannot credibly profess to understand unspoken social norms if your appearance contradicts your words.