For those of us with more peculiar tastes, this isn't actually true.

Good point. Personally, I'm the same as you - my tastes are particular enough where I've never even met anyone who I've felt compatible with. But I think that tens of thousands is a reasonable estimate for the general population.

I'm in college right now where the concentration of such men is the highest it will ever be in my life

I don't think that that's true. I'm in college now too (Pitt), and people all seem very average to me. I think that for you, it's more likely to meet people you're looking for by doing things you're interested in (rationality and transhumanist stuff), and you'll probably do more of that stuff once you graduate.