To jump off of what komponisto said, I had a similar reaction when I was at Yale also. Now that I'm a grad student at a less prestigious university I see a lot of dumb undergrads as well as dumb people in some other departments. It seems that the smartest kids at BU are as smart as the smartest Yalies. But there are a lot more dumb kids and the dumbest undergrads are much dumber than the dumbest Yalies.
What do you believe because others believe it, even though your own evidence and reasoning ("impressions") point the other way?
(Note that answers like "quantum chromodynamics" don't count, except in the unlikely case that you've seriously tried to do your own physics, and it suggested the mainstream was wrong, and that's what you would have believed if not for it being the mainstream.)