Many of the high school and college students who contacted us at Cognito Mentoring were looking for advice were considering going into academia. The main draw to them was the desire to learn specific subjects and explore ideas in greater depth. As a result, we've been investigating academia as a career option and also considering what alternatives there may be to academia that fulfill the same needs but provide better pay and/or generate more social value. The love of ideas and epistemic exploration is shared by many of the people at Less Wrong, including those who are not in academia. So I'm hoping that people will share their own perspectives in the comments. That'll help us as well as the many LessWrong lurkers interested in academia.
I'm eager to hear about what considerations you used when weighing academia against other career options, and how you came to your decision. Incidentally, there are a number of great answers to the Quora question Why did you leave academia?, but there's probably many thoughts people have here that aren't reflected in the Quora answers. I've also written up a detailed review of academia as a career option on the info wiki for Cognito Mentoring here (long read), and I'd also love feedback on the validity of the points I make there.
Many of our advisees as well as the LessWrong readership at large are interested in choosing careers based on the social value generated by these careers. (This is evidenced in the strong connection between the LessWrong and effective altruism communities). What are your thoughts on that front? Jonah and I have collaboratively written a page on the social value of academia. Our key point is that research academia is higher value than alternative careers only in cases where either the person has a chance of making big breakthroughs in the area, or if the area of research itself is high-value. Examples of the latter may include machine learning (we're just starting on investigating this) and (arguably) biomedical research (we've collected some links on this, but haven't investigated this in depth).
For those who are or were attracted to academia, what other career options did you consider? If you decided not to join, or chose to quit, academia, what alternative career are you now pursuing? We've identified a few possibilities at our alternatives to academia page, but we're largely shooting in the dark here. Based on anecdotal evidence from people working in venture capital, it seems like venture capital is a great place for polymath-types who are interested in researching a wide range of subjects shallowly, so it's ideal for people who like shallow intellectual exploration rather than sticking to a single subject for an inordinate amount of time. But there are very few jobs in venture capital. On paper, jobs at consulting firms should be similar to venture capital in requiring a lot of shallow research. But we don't have an inside view of consulting jobs -- are they a good venue for intellectually curious people? Are there other job categories we missed?
All thoughts are greatly appreciated!
LessWrong has a relatively strong anti-academic bias, and I'm worried that this is reflected in the comments.
I work as a PhD student in machine learning, and yes, there is a minimum bar of intelligence, perseverance, etc. below which doing high-quality research is unlikely. However, in my experience I have seen many people who are clearly above that bar who nevertheless go into industry. This is not to say that their choice is incorrect, but on balance I think the argument "don't go into academia unless you'll be one of the smartest people in your field" does more harm than good. It also seems to me that the effective altruist movement, in particularly, mostly overlooks academia as an altruistic career option, even though I personally think that for many intelligent people (including myself), working on the right research problems is the most valuable contribution they can make to society.
If you go into a field like mathematics or theoretical physics, yes, you're unlikely to make a meaningful contribution unless you're one of the best people in the field. This is because these fields have basically become an attractor for bright undergrads looking to "prove themselves" intellectually. I'm not trying to argue that these fields are not useful; I am trying to argue that the marginal usefulness of an additional researcher is low barring extraordinary circumstances.
In other fields, especially newer fields, this is far less true. Machine learning has plenty of low-hanging fruit. My impression is that bioinstrumentation and computational neuroscience do as well (not to mention many other fields that I just don't happen to be as familiar with). This is not to say that working in these fields will be a cake-walk, or that there isn't lots of competition for faculty jobs. It is to say that there are huge amounts of value to be created by working in these fields. Even if you don't like pure research as a career option, you can create huge amounts of value by attaching yourself to a good lab as a software engineer.
It's also worth noting that "doing research" isn't some sort of magic skill that you do or don't have. It's something you acquire over time, and the meta-skills learned seem fairly valuable to me.
How do you know this? Have there been a lot of findings made by a lot of people without any indication that this stream of discoveries is slowing down? When I looked up e.g. Deep learning it seemed to be a relatively old technique (1980's and early 90's). What are some examples of recent discoveries you would describe as low-hanging fruits?