In undergrad I feared a feeling of locked-in-ness, and ditched my intention to do a PhD in math (which I think I could have done well in) partly for this reason, though it was also easier for me because I hadn't established close ties to a particular line of research, and because I had programming background. I worked a couple of years in programming, and now I'm back in school doing a PhD in stats, because I like probability spaces and because I wanted to do something more mathematical than (most) programming. I guess I picked stats over applied math partly out of the same worry about overspecialization; I think stats has a bigger wealth of better-integrated more widely applicable concepts/insights.
I am curious: what do you plan to work on in stats?
I personally think more people should be working on efficient general sampling methods for Bayesian stats, for reasons I have written about here: http://goodmorningeconomics.wordpress.com/2010/11/16/the-promise-of-bayesian-statistics-pt-2/ .
Programming skills are very useful there. I am a programmer and one of my hobbies is implementing bayes stats algorithms in the literature. Do let me know if you come up with anything revolutionary.
Currently I'm taking classes and working on a polytope sampler. I tend to be excited about Bayesian nonparametrics and consistent families of arbitrary-dimensional priors. I'm also excited about general-purpose MCMC-like approaches, but so far I haven't thought very hard about them.
It's just a vanilla (MH) MCMC sampler for (some convenient family of) distributions on polytopes; hopefully like this: http://cran.r-project.org/web/packages/limSolve/vignettes/xsample.pdf , but faster. It's motivated by a model for inferring network link traffic flows from counts of in- and out-bound traffic at each node; the solution space is a polytope, and we want to take advantage of previous observations to form a better prior. But for the approach to be feasible we first need to sample.
But this is not a long-term project, I think.
I'm a mathematician, and I had similar problems in the past. Here are some things that have worked fantastically well for me.
First of all, just start talking to people in other fields at your university. Start with "nearby" fields (like theoretical physics) and progress from there. See what they're up to; you might find opportunities for fruitful collaboration. These people need not be professors; indeed, since they're used to being pestered by eager students trying to climb the social ladder, you might find professors unable or unwilling to give you their time unless you can convince them it'll be to their benefit. Fortunately, talking to students can often be just as worthwhile.
Other things you can do: read first-year textbooks. In the fields that you find interesting, read more advanced textbooks. This can be highly enjoyable in its own right; I like to savour the different kinds of argumentation and notions of salience that get used in different fields. You can also attend conferences and that sort of thing.
If you're interested in escaping the bounds of academia, of course, you should also talk to people who aren't academics. You'll still want to find people doing something worthwhile, of course: I find I get along best with artists, radical/activist types, social workers, etc.
One caveat in all this: approach unfamiliar beliefs and methodologies with openness and respect. Often it takes awhile to really grok why things are done in certain ways, and moreover, people tend to take offense (rightly!) at newbies who are too quick to attack what they don't yet understand. If something seems silly or irrational to you, it may very well be; but people are on the whole more rational than the LW crowd tends to give them credit for, and so it's just as likely that you simply aren't familiar yet with their motivations. Ask lots of questions, but ask them with the appropriate attitude, and actually listen carefully to the answers you get.
I got an undergrad degree in math and realized that I needed to do a lot of maturing if I were to succeed in grad school. So I got a job which locked me in for 3 years (due to the intricacies of vesting), and since I work nights, I eventually decided that I needed a house so I could actually get sleep, which has locked me in to my situation for at least 3 more years (because of a requirement of the housing credit).
So yeah, I am currently stuck right where I am.
However, my job is at the university, which means that I can take free classes. That's how I learned about income tax. I tried to jump into stats grad classes, but I think taking Bayesian Statistics at the same time as its prerequisites was a bit too much to handle at once. I'm now working on getting a CS degree, since the Bayes class is only offered every other aeon. I also decided that I had too little practical knowledge, so I took up a metal smithing class, and do a little construction on the side in the summers. Thus, while I am nominally stuck, I have steadfastly refused to stick to one thing.
I have no real idea what I'm going to do with my life. I don't feel like I have Something To Protect,, which bothers me, but I have a strong urge to Become Stronger. My decisions have been without long-term aim. Even as an undergrad, I switched between majors from CS to Psych to History before settling on math, mostly because I felt that it would close the fewest future doors.
So in short, I have been guided by a sense of biding my time, trying to gain financial resilience, and trying to help friends and family also gain resilience. I have yet to decide what I will be doing.
I don't feel locked in, but then... physics.
Most fields really aren't that inaccessible to curious, intelligent people. I suspect, though, that you don't have anything other than math that you're particularly interested in.
To survey the wide world for things you're interested in, I'd recommend reading popularizations of science (either in book form or magazine form). Oh, and attend public lectures, they're fun.
Or you could play the "What do I want, what do I have, how do I use what I have to get what I want?" game a lot. Maybe with multi-leveled diagrams.
I've had the "locked in" feeling in general, manifesting as the feeling that my life will be significantly less optimal if I change anything.
Then one day a friend pointed out that this may just be an indication I've made an approximately optimal set of life choices.
So now that feeling doesn't bother me.
I know multiple instances of math majors switching to non-math fields and easily getting good positions. I myself was doing fairly good research (improving on the current best technique for solving a problem that people care about) within 8 months of switching from math to robotics (although I also spent a year doing computational cognitive science in the time between math and robotics).
It sounds like the real issue is figuring out what field to go into. Personally I just tried a few different fields until I started to get a better idea of what was going on, and finally was able to narrow down to a shortlist of problems that I consider really important (probably similar to paulfchristiano's, it looks something like synthetic biology, brain scanning / simulation, machine learning, environmental engineering, clean energy, and materials science).
If you ask yourself what you would like the world to look like, it is usually easy to at least differentiate between things that do a lot to move you towards that goal, and things that do very little; maybe at the top it is hard to figure out which research programs will be the most effective, but it is probably not difficult to find something that is more effective than what you are currently doing, which should be good enough from the perspective of deciding to switch.
You are right in that the real issue is figuring out what field to go into.
The thing about it is that I don't really know how to taskify "trying a few different fields;" this is perhaps the soul of my question.
Find a professor at your university that works in a non-math field but is still mathematically sophisticated enough not to bore you. Then take their class (assuming that it focuses on their research rather than being some sort of introductory-level class). Possibly you are unable to figure out if someone satisfies these criteria. If you're willing to reveal what university you are at, and link to its course catalog, I can throw out suggestions.
I was in a PhD program for Physics, then left it for Operations Research (where I will likely get a masters then exit, but may stick around for the PhD). In some sense that is the field of applied rationality, so you might want to check it out (it may be called Industrial Engineering where you are).
Did I feel locked in? A little. I was leery about committing to a research group for several years, as I hadn't come across any groups that I was really interested in or ideas I really wanted to pursue. But I always had very broad interests (I have programming skills, employment experience, finance knowledge, and know a fair amount about math), and so while I eventually realized I was sticking with my plans out of inertia I had several other options that I was ready to jump ship for.
I ended up deciding on OR rather than economics much for the same reason I decided on physics over economics- economics is what I get most fired up about (well, efficiency in general, but it's more fun to argue than do), but I wanted to make something besides policy papers. I was already planning to take my physics degree and do industrial research / invent something / go into consulting, and so I decided to change my "invent something" desire from physical things to software and scrap the industrial research plan in order to get into consulting in 2 years rather than 5-6.
You may want to look at being an actuary. They draw a rather significant income using skills you have most likely already developed.
Becoming an actuary is something that I've looked into in the past. I was turned off by the various certifications that I didn't understand, and the fact that I have essentially zero finance knowledge.
But when I looked at the actuarial exams, I was like "word problems?! HELLS YES!" so it's definitely something that is on my radar.
When I said "most fields" I really meant "almost every field" (including math).
There are a lot of problems that need solving which will still get you publications, even in math. The issue is that there are also a lot of problems which don't need solving which will nevertheless get you publications (and that telling which are which is difficult). Given the number of people who seem to be somehow engaged with theoretical research here, it seems like a topic that could be profitably discussed.
I suspect that there are. However, I doubt my ability to accurately find such problems and solve them.
I am in particular interested in finding non-math things that I am interested in and could reasonably pursue, and narrowing down such "non-apples" to something like this or this (the part about biotech); and then especially I'm interested in turning that into a set of tasks.
What I would like to leave with is a to do list that reads like: "Take chemistry classes next year and earn a degree in biology while remaining a graduate student, then apply for biotech internships."
Obviously this is somewhat personal and I've digressed into something that is more of an addendum to the OP rather than a reply to your comment.
However, I doubt my ability to accurately find such problems and solve them.
I am not convinced that this gets too much easier in other fields. For example, in biology, a natural and apparently rather straightforward problem (compared to some of the long term goals of the field) is killing all humans. Its not clear how the field should proceed to accomplish something useful without introducing a serious existential risk. If you feel unable to evaluate the long-term impacts of your work, it seems possible to do a great deal of harm, nevermind doing no good.
I guess my impression is that determining which mathematical problems are worth solving is more abstract and difficult than determining which problems need to be solved in, for example, biology. That is, it is obvious to me that "develop new antidepressants" is a better decision than "kill all humans," whereas "develop new factorization algorithm" may or may not be a better decision than "use group theory to study certain differential equations."
Obviously there is a problem here of specificity; more technical decisions in biology may be equally hard, but it is in general hard to reduce problems in mathematics to external applications.
Also, I got the impression that killing all humans was pretty much a solved problem. Fortunately, the solution has not yet been implemented.
Also, I got the impression that killing all humans was pretty much a solved problem. Fortunately, the solution has not yet been implemented.
The real question is how easy it is. Requiring a significant coordinated effort by a major lab is one thing (though I don't even think we are there yet)---requiring one particularly careless guy with $100,000 is another.
Sure. "Killing all humans" is solved in the sense that "factoring large integers" is solved.
We can do it in O(3+log(x)/log(phi)) time, but can we do it faster?
Response to: The Value of Theoretical Research
Reading paulfchristiano's article the other day, I realized that I had had many similar discussions with myself, and have been guilty of motivated stopping and poor answers to all of them.
However, one major roadblock in my pursuing better answers, is that I feel that I have been "locked in" to my current path.
I am currently a mathematics Ph.D. student. I did not have a minor. I don't have significant programming skills or employment experience. I know nothing about finance. I know a lot about mathematics.
Paul says:
However, "most fields" is not a very good tool for narrowing my search space; I have spent my entire life in school, and I like having structures and schedules that tell me when I'm doing productive things and that I've progressed to certain stages. I'm not ready to drop out and do whatever, and I don't have a particular idea of what whatever might be.
On the other hand, I currently have a variety of resources available to me. For example, I have a steady income (a grad student stipend isn't much, but it's plenty for me to live on), and I have the ability to take undergraduate classes for free (though not the spare time at the moment.)
My current intent is to continue and finish my Ph.D., but to attempt to take classes in other subjects, such as linguistics, biology and chemistry, and computer science which might lead to other interesting career paths.
Has anybody else had a similar feeling of being "locked in"? How have you responded to it? For those who have studied mathematics, are you still? If you continued, what helped you make that decision? If you stopped, what about that? What did you end up doing? How did you decide on it?