If you want a faculty job badly enough you can get one.
I disagree. I think much of the evidence about the rise of post-docs as principal investigators and the diminishing number of tenured positions is at odds with this claim. This claim is essentially why most students go to a Ph.D. program and they become depressed when they learn it doesn't work like this about 3 years into the process.
For instance, if you haven't traveled a lot in the past 5 years you probably don't find travel all that important (otherwise you would've found a way to travel).
In the last 5 years, I've taken low-paying math research jobs several summers so that I could live in Paris, Hong Kong, and College Station, TX, just to experience parts of the world I had not been to. I've moved (at great personal expense) 3 times in the past 4 years to get out of life situations that I found unsatisfactory. I think that my thinking-to-action ratio is not bad.
You seem to dismiss the possibility that there can be real life Catch-22s. Given my preferences, I think I am in a Catch-22 and I cannot determine an actionable step. Some of my favorite life advice came from a high school math teacher who said "when you don't know what to do, do something." I think I am more insightful than just to wallow in akrasia.
So if you go systematically through the papers published in the last year or so you'll find either (a) all the papers are boring, stupid, silly or wrong.
Yes, this is exactly what I have been doing for the past 2 years. But when I have discussed the option to switch to other research fields with faculty and older graduate students, they are telling me that the condition (a) is going to be true in every research field where there is actually enough grant money to finance my studentship, and that (a) is just a part of life in science and that I should be more focused on just doing programming tasks and coming up with small software developments that cater to commercial interests, leading to papers that fit into condition (a). I completely reject their point of view; I think they are wrong, and I think that if academia is set up this way, then my options are to leave academia for jobs that I think are very suboptimal or else agree to unhappily suffer through the academic hoop-jumping that I don't like.
Given that these are my only options, I am trying to prepare myself to choose one or the other. But the biggest opportunity cost that I feel scared about is losing the chance for theoretical research and philosophical aspects of science to be a major component of the value that I contribute over my career.
I've posted a few things seeking career advice with mixed success. In this case I have a more concrete question and if you feel like commenting, I'd appreciate it. I think it helps me to hear what a community of others thinks from a rational perspective because there are often many components to a decision that I had not anticipated.
I am currently a grad student working in computer vision. I dislike the way that my current adviser focuses only on projects that have short-term commercial gains. I want to study more fundamental, theoretical research which may take more time to develop but will also be more aesthetically pleasing to me. For me, the only reason to agree to be paid so little as a graduate student is to gain the opportunity to work freely on high risk projects that happen to be of personal interest. Practical considerations are not interesting to me as motivation for a Ph.D. On the other hand, it has felt nearly impossible to actually find faculty willing to have students work on theory. Rather than grinding away with no dental insurance for 3 more years, followed by low paying post-docs, etc., perhaps seeking a job will be better.
I have some interesting job prospects that are all with larger companies. The jobs are basically business analytics, including scientific computing, data mining, and machine learning. I'm sure the problems to work on are not that great; not going to be Earth shattering, but at the same time they sound a lot more interesting to me than hedge fund data analysis or military research labs (I have working experience at a government lab and I did not enjoy it). The hours would be better; the pay is fair and it would be a good living. I could pursue some things as serious hobbies outside work.
At the same time though, there feels like a nagging opportunity cost. I am not naive enough to believe there will be a nice faculty job waiting for me even if I finish my Ph.D. However, I really enjoy theoretical and mathematical physics, machine learning, computational complexity, and scientific computing, and various philosophical considerations generated by these. Being able to teach about them, research them, and work on them professionally seems incredibly appealing. Am I making a big mistake if I leave? How can one pursue philosophical interests and desires to work in theory outside of a typical job? Or should I even worry about such a thing?