So my question: what attracts smart and curious young people to physics?

One of the reasons I went into physics (for undergrad, at least) instead of biology or chemistry was that my comparative advantage was higher at physics. There was very little memorization involved- learn the laws, apply the laws, and you're done. If you're clever and good at algebraic manipulation, it doesn't require that much work, whereas biology of chemistry have massive amount of facts that must be stored somehow.

That doesn't explain the preference over computer science or math- neither of those seem memorization intensive- but might explain why you're looking at that set of three, instead of a set of five.

Is there some sort of mood affiliation going on here, where the smartest people are pulled to physics to distinguish themselves from the crowd, insofar as physics is more difficult and repels the crowd?

Quite possibly. I would suggest to any smart young person that if they're going to college, they should try for an undergrad degree in physics, primarily because of the high-quality classmates (and the intellectual training is nice too). With AP exams, double majors are fairly easy to do, and so doing both physics and whatever you find interesting seems like a strong option.