First, I think you are making an unfair comparison- sure, upper level physics classes like quantum mechanics are unlikely to help in your everyday life. However, upper level mathematics are similarly useless. Quantum mechanics is as good a place as any to cement the concepts of linear function spaces. General relativity is as good a place as any to learn differential geometry, etc. If you stop at undergrad, the mathematician and the physicists have basically overlapping skillsets (often physics majors double major in math because its so few additional courses).
The same idea (intro is useful, beyond that diminishing returns) is true of almost any discipline. Intro microeconomics is fairly useful,but DSGE modeling is essentially useless outside of specialized career tracks.
I work as a data scientist/statistician, and have a phd in theoretical physics. I would say that physicists seem much more capable of handling actual data and making interesting statements about it than mathematicians, at least at the post-phd level. I would say the generalized skill of "engaging with data" is the single most useful skill my phd left me with, but thats at least in part because of my work life.
Also, I've found that if you build things, physics insight is absolutely invaluable, and I think that many smart people have an interest in building cool things. I recently designed and built a stirling engine to turn my window fan. Previous projects include a small zip gun (to settle an argument with a guy about how useless current 3d printing is), three go carts armed with nerf-projectiles that temporarily deactivate your enemies go-cart when struck, a small programmable mouse for my cat to play with, a lighted "dancing" fountain for the backyard. For the fountain, I designed a small solenoid valve. None of this required anything beyond a good intuition for classical physics (em, thermo, etc), but I've found that intuition for classical physics invaluable.
Many of the high school students who sought advice from Cognito Mentoring were interested in mathematics, computer science, and physics. This both makes sense and is valuable. Mathematics has many benefits: it underpins a lot of quantitative analysis, and helps us understand the world. Computer science is also quite important for obvious reasons: programming in particular is directly and indirectly useful, and a deeper understanding of algorithms and the theory of computation can help with algorithms.
Physics, however, is a little different. There are some benefits of learning physics. In particular, classical mechanics is often people's first exposure to using mathematical structure in a nontrivial way to understand and model situations pertaining to the real world. Nonetheless, unlike mathematics or computer science, the benefits of physics for people who are not in science or engineering careers are fairly low. I find myself using high school-level mathematical intuition on a regular basis (for instance, understanding the growth trajectories of various things, or interpreting graphs), and I find myself using basic programming-like intuition quite often. But I rarely find myself using my physics intuition in the real world. Moreover, I think physics quickly hits diminishing returns in terms of teaching people about mathematical modeling: I'd say that the returns from physics beyond classical mechanics, DC circuits, and basic thermodynamics are near-zero. For instance, I'd say it's more beneficial to learn microeconomics rather than electromagnetism, even though the latter is often considered more prestigious by smart people. Similarly, I think that behavioral economics is more valuable than quantum mechanics.
It's also not clear that learning physics beyond the basics suggested above (classical mechanics, thermodynamics, DC circuits) passes a cost-benefit analysis for people in the vast majority of science-based and engineering-based careers. Even the extent to which they crucially rely on these basics is questionable, given that most people don't learn the basics well and still manage to go on to do decent jobs. I'd like to hear any opinions on this. On a related note, I recently asked on Quora the question In what ways is knowledge of Newtonian classical mechanics helpful to people pursuing biomedical research? and there were a few interesting answers.
So my question: what attracts smart and curious young people to physics? Are the smartest people too attracted by physics, relative to its real-world applicability? Does the intellectual stimulation provided by physics justify the attraction? Is there some sort of mood affiliation going on here, where the smartest people are pulled to physics to distinguish themselves from the crowd, insofar as physics is more difficult and repels the crowd? To the extent that people overvalue physics, does it make sense to push them at the margin away from physics and in the direction of computer science or economics or some other subject? Or should their interest in physics be encouraged?
Thoughts on your personal experience, as well as thoughts on the general points about the usefulness and attractiveness of learning physics, would be appreciated.
PS: In a video, Eric Mazur describes research related to the Force Concept Inventory: people often learn how to solve complicated mechanics problems by pattern-matching but fail to demonstrate clear understanding of Newton's Third Law. Similarly, people can predict potential differences and current flows in complicated circuits using Kirchhoff's laws, yet fail to predict that if you short a circuit, all the current will flow through the short. (The latter failure of prediction occurred in an end-of-course examination co-taught by Mazur to Harvard University first-year students, many of whom were planning to go on to medical school.
PS2: My collaborator Jonah Sinick's Quora post (no login needed to view) titled Is math privileged for gifted children is somewhat related.