For many of us, choosing a career path has a dominant effect on our contribution to the society. For those of us who care what happens to society, this makes it one of the most important decisions we make. Like most decisions, this one is very often made by impulses significantly below the level of conscious recognition, with considerable intellectual effort spent on justifying a conclusion but very little spent on actually reaching one. In the case of smart, altruistic rationalists, this seems like the most tragic failure of rationality; so, whatever the outcome, I advocate much more serious consideration by smart rationalists of how our career choices affect society. For the most part this is a personal thing, but some public discussion may be valuable. I apologize (largely in advance) for anything that seems condescending.
I previously planned to do research in pure math (and more recently in theoretical computer science). I frequently justified my position with carefully constructed arguments which I no longer believe. It still may be the case that doing research is a good idea (and spending the rest of my life doing research is still the easiest possible career path for me), so I am interested in additional arguments, or reasons why anything I am about to write is wrong. Here is a basic list of my justifications, and why I no longer believe them.
Argument 1: Much math is practically important today. The math I am working on is not practically important today, but maybe it will be the math that is practically important tomorrow. How can we predict what will be useful? It seems like pushing math generally forward is the best response to this uncertainty.
Rebuttal: If we really want to evaluate this argument, it is important to understand the conditions under which the important math of today was done. In the case of calculus, differential equations, statistics, functional analysis, linear algebra, group theory, and numerical methods, the important results for modern work were in fact developed after their usefulness could be appreciated by an intelligent observer. There is very little honestly compelling evidence that pushing math for the sake of pushing math is likely to lead to practically important results more effectively than waiting until new math is needed and then developing it. Perhaps the most compelling case is number theory and its unexpected application to cryptography, which is still not nearly compelling enough to justify work on pure math (or even provide significant support).
Argument 2: Math is practically important today. The math I am working on is in a field that is practically important today, and not many people are qualified to work on it, so pushing the state of the art here is an excellent use of my time.
Rebuttal: Consider the actual marginal utility of advances in your field of choice, honestly. In the overwhelming majority of cases, the bulk of research effort is directed grotesquely inefficiently from a social perspective. In particular, a small number of largely artificial applications will typically support research programs which consume an incredible amount of intelligent mathematicians' time, compared to the time required to make fundamental progress on the actual problem that people care about. Here you have to make a different argument for every research program, which I would be happy to do if anyone offers a particular challenge.
Argument 3: Theoretical physics research advances the fundamental limits of understanding, which has led to important advances in the past and will probably continue to lead to important advances.
Rebuttal: What matters are interactions in regimes that humans can engineer---improving understanding in such regimes is responsible for every technological development I am aware of. In particular, improvements in our understanding of high energy physics or cosmology are unlikely to be useful until we can design systems which operate in those regimes. There is fundamental physics research which seems likely to have a high payoff---but if you approach theoretical physics with the honest goal of contributing to technological progress, you end up with a research program which is unrecognizably different from most physicists'.
Argument 4: Pure research is at least a little useful, and its what I am best prepared to do.
Rebuttal: There is a shortage of intelligent, rational people in pretty much every area of human activity. I would go so far as to claim this is the limiting input for most fields. If you don't believe this, at least ask yourself why not. Do you have experience in other fields that suggests you are unable to contribute? Do you have a causal argument?
Argument 5: Society is relatively efficient. The marginal returns for work in every field are roughly comparable, so I should work wherever I have comparative advantage.
Rebuttal: Why should society be remotely efficient? I believed this for a long time, but eventually realized it was just a hold-over from a point in my life where I had more faith in other people. If you are typical LW readers, you probably believe at least half a dozen strong counterexamples to this claim already.
Argument 6: Pure research has fundamental value as an intellectual pursuit.
Rebuttal: For whom? If you are concerned exclusively with the intellectual richness of mathematicians' lives, then I can't well disagree and this argument may be completely convincing. Otherwise, if you believe that the increasing richness of human mathematics is a fundamental good which non-mathematicians can enjoy, consider the inferential distances separating modern advances from even the most intelligent layperson. If your ultimate goal is the production of mathematics, or in fact any temporally altruistic objective, then consider alternatives which may increase the future's capacity to do mathematics and which may be orders of magnitude more effective.
Argument 7: What else would I do to make a living? Research provides at least some benefit to society; alternatives seem even worse.
Rebuttal: My past self, at least, was guilty of motivated stopping. See argument 4.
I'm getting a PhD in math and I have had similar thoughts. A few quick remarks about your specific arguments followed by my personal take:
In rebuttal 1, you mention number theory as an example where the application to crypto took a long time to be apparent. I'm a number theorist and I don't find this argument compelling. Much of the number theory that is used in crypto is not deep. There's nothing about Diffie-Hellman or RSA that requires the hundreds of years of number theory research that has gone into it. One could explain the algorithms for such procedures to a mathematician in the early 1800s with little effort (although the idea of having very efficient methods of arithmetic might strike them as very odd). Moreover, while other parts of number theory have turned out to be relevant it is still a very tiny fraction of all number theory.
A possibly better example would be non-Euclidean geometries which really were studied in detail in the 19th century before they were found to have practical applications.
As for myself, I enjoy math a lot, and I suspect that I will be more productive at areas I enjoy than areas I enjoy less. This might be a rationalization but it connects to another aspect: I'm not a good utilitarian. Given the choice between me having a happy life and being somewhat productive for humanity and given the choice between being less happy and more productive for humanity I'll choose being more happy. (When I phrase it that way it triggers far more reactions that lead me to not want to do that. But what decisions I make on a day to day basis about what to think about and what to do with my time are very much near mode).
I feel the same way you do, including the last paragraph, with a possible exception: I do not endorse the fact that I am a bad utilitarian.
Because of this I'm currently attempting to invoke a crisis moment in which I can effectively change fields, although I suspect that I will stay in graduate school to get my Ph.D. and use the opportunity to get more of an education.