There are a few issues with this. First, it is not at all true that CS degree holders can skip the post-doc. There is a lot of evidence that post-docs are rising as principal investigators in research grants and that, as most universities reduce the number of tenured positions available, they are supplementing the workload with mostly post-doc positions because they can be paid less and are entitled to fewer benefits. In fact, when I worked at MIT Lincoln Laboratory, it was actually an openly discussed part of their business model that hiring people as post-docs was more cost effective since they could hire 2 or 3 post-docs for the same price as a single full-hire at the PhD level. Because their government projects had a turnaround time of about 2-3 years, it was a perfect fit and even if the post-docs left the lab, there was not any serious corporate brain-drain going on. A few permanent engineers plus a constantly churning team of 20 post-docs could easily crunch through years worth of government projects without losing much to the high employee turnover. From what I understand, Los Alamos and Sandia National Labs are also adopting this business model. Essentially, positions are offered as being "prestigious post-doctoral fellowships" that pay ~40k/year for a job that most other employers would have to pay 70-100k/year for. I am even starting to see this in private corporations, such as Kodak and IBM where they also have rotating systems of post-docs instead of hiring long-term employees. One part of this is just that with the rise of certain technologies and with an increasingly risk-averse stance toward basic science research, the wages for skilled scientific workers are just plummiting.
The other thing, which is mentioned in this Economist article, is that in mathematics and computer science, someone who leaves a PhD program early with only a master's degree will earn exactly the same wage as someone with a PhD, on average. Coupled with the fact that the master's degree holder will have a 2-4 year lead time working with a company, they will be closer to a promotion, have more years of experience, and more years of retirement savings. Over the lifetime of a career, stopping with only a master's degree will allow you to earn more money than if you finish the PhD, and this is across all work sectors.
So basically, my dilemma is this: if I am just going to be forced to take one of these industry kind of research jobs, then it is economically more rational for me to just stop with a master's degree and begin working now. The number of hours I work as a student is severe, so taking a regular 9-5 job would lower my stress level, give me health insurance, increase my financial situation, and allow me some amount of time to pursue hobbies outside of work. Currently, in school, all of those areas are bad for me.
After working for two years as an algorithm developer for an air force lab, I really dislike the military/government research sector and I want to avoid working there if possible. I have some small interest in financial mathematics, and could probably get a job at a hedge fund, but I am not sure I will really be happy in that stressful situation. Having money in the short term has never been a truly significant motivator for me. Having lots of money in the long term is more motivating because I could then be an investor who finances PhD studentships myself and potentially I could commission more work to be done in the basic science research that I think is being ignored across academia today. But is it worth working doggishly through my young life to potentially be a wealthy investor later on? I don't know.
This leaves me feeling like my only real industry options would be for corporations, some of which are just as bad as government labs. I have found some interesting options in the entertainment / web development domain that, even though they do not involve mathematical theory or deeply intellectual pursuits, do combine machine learning, scientific computing, and seem to be places where a skilled mathematician could succeed in ways that typical workers for these jobs could not.
I'm just trying to decide whether committing myself to jump ship out of a PhD is really worth it if one of my main long-term preferences is to "contribute to basic science research." Really, my goal might be better stated as "become the Radiohead of science" and start my own version of something like github, but instead of software development, it would be for farming out small actionable steps in larger scientific research projects. Do I really, truly believe this is something I can do in my spare time while pursuing career success for an entertainment company, software firm, etc.?
I've posted a few things seeking career advice with mixed success. In this case I have a more concrete question and if you feel like commenting, I'd appreciate it. I think it helps me to hear what a community of others thinks from a rational perspective because there are often many components to a decision that I had not anticipated.
I am currently a grad student working in computer vision. I dislike the way that my current adviser focuses only on projects that have short-term commercial gains. I want to study more fundamental, theoretical research which may take more time to develop but will also be more aesthetically pleasing to me. For me, the only reason to agree to be paid so little as a graduate student is to gain the opportunity to work freely on high risk projects that happen to be of personal interest. Practical considerations are not interesting to me as motivation for a Ph.D. On the other hand, it has felt nearly impossible to actually find faculty willing to have students work on theory. Rather than grinding away with no dental insurance for 3 more years, followed by low paying post-docs, etc., perhaps seeking a job will be better.
I have some interesting job prospects that are all with larger companies. The jobs are basically business analytics, including scientific computing, data mining, and machine learning. I'm sure the problems to work on are not that great; not going to be Earth shattering, but at the same time they sound a lot more interesting to me than hedge fund data analysis or military research labs (I have working experience at a government lab and I did not enjoy it). The hours would be better; the pay is fair and it would be a good living. I could pursue some things as serious hobbies outside work.
At the same time though, there feels like a nagging opportunity cost. I am not naive enough to believe there will be a nice faculty job waiting for me even if I finish my Ph.D. However, I really enjoy theoretical and mathematical physics, machine learning, computational complexity, and scientific computing, and various philosophical considerations generated by these. Being able to teach about them, research them, and work on them professionally seems incredibly appealing. Am I making a big mistake if I leave? How can one pursue philosophical interests and desires to work in theory outside of a typical job? Or should I even worry about such a thing?