I was initially going to title this post, "try to become an AI researcher and, if so, how?" but see Hold Off On Proposing Solutions. So instead, I'm going to ask people to give me as much relevant information as possible. The rest of this post will be a dump of what I've figured out so far, so people can read it and try to figure out what I might be missing.
If you yourself are trying to make this decision, some of what I say about myself may apply to you. Hopefully, some of the comments on this post will also be generally applicable.
Oh, and if you can think of any bias-avoiding advice that's relevant here, along the lines of holding off on proposing solutions, that would be most helpful.
Though I'm really hard to offend in general, I've made a conscious decision to operate by Crocker's Rules in this thread.
One possibility that's crossed my mind for getting involved is going back go graduate school in philosophy to study under Bostrom or Chalmers. But I really have no idea what the other possible routes for me are, and ought to know about them before making a decision.
~~~~~
Now for the big dump of background info. Feel free to skip and just respond based on what's above the squigglies.
I seem to be good at a lot of different things (not necessarily everything), but I'm especially good at math. SAT was 800 math, 790 verbal, GRE was 800 on both, but in both cases, I studied for the test and getting my verbal score up was much harder work. However, I know there are plenty people who are much better at math than I am. In high school, I was one of the very few students from my city to qualify for the American Regions Math League competition (ARML), but did not do especially well.
Going to ARML persuaded me that I was probably not quite smart enough to be a world-class anything. I entered college as a biochemistry major, with the idea that I would go to medical school and then join an organization like Doctors Without Borders to just do as much good as I could, even if I wasn't a world-class anything. I did know at the time that I was better at math than biology, but hadn't yet read Peter Unger's Living High and Letting Die, so "figure out the best way to covert math aptitude into dollars and donate what you don't need to charity" wasn't a strategy I even considered.
After getting mostly B's in biology and organic chemistry my sophomore year, I decided maybe I wasn't well-suited for medical school and began looking for something else to do with my life. To this day, I'm genuinely unsure why I don't do so well in biology. Did the better students in my classes have some aptitude I lacked? Or was it that being really good at math made me lazy about things that are inherently time-consuming to study, like (possibly) anatomy?
I took a couple of neuroscience classes junior year, and considered a career in the subject, but eventually ended up settling on philosophy, silencing some inner doubts I had about philosophy as a field. I applied to grad school in philosophy at over a dozen programs and was accepted in to exactly one: the University of Notre Dame. I accepted, which was in retrospect the first or second stupidest decision I've made in my life.
Why was it a stupid decision? To give only three of the reasons: (1) Notre Dame is a department where evangelical Christian anti-evolutionists like Alvin Plantinga are given high status (2) it was weak in philosophy of mind, which is what I really wanted to study (3) I was squishing what were, in retrospect, legitimate doubts about academic philosophy, because once I made the decision to go, I had to make it sound as good as possible to myself.
Why did I do it? I'm not entirely sure, and I'd like to better understand this mistake so as to not make a similar one again. Possible contributing factors: (1) I didn't want to admit to myself that I didn't know what I was doing with my life (2) I had an irrational belief that if I said "no" I'd never get another opportunity like that again (3) my mom and dad went straight from undergrad to graduate school in biochemistry and dental school, respectively, and I was using that as a model for what my life should look like without really questioning it (4) Notre Dame initially waitlisted me and then, when they finally accepted me, gave me very little time to decide whether or not to accept, which probably unintentionally invoked one or two effects straight out of Cialdini.
So a couple years later, I dropped out of the program and now I'm working a not-especially-challenging, not-especially-exciting, not-especially-well-paying job while I figure out what I should do next.
My main reason for now being interested in AI is that through several years of reading LW/OB, and the formal publications of the people who are popular around here, I've become persuaded that even if specific theses endorsed by the Singularity Institute are wrong potentially world-changing AI is close enough to be worth putting a lot of thought into seriously thinking about.
It helps that it fits with interests I've had for a long time in cognitive science and philosophy of mind. I think I actually was interested in the idea of being an AI researcher some time around middle school, but by the time I was entering college I had gotten the impression that human-like AI was about as likely in the near future as FTL travel.
The other broad life-plan I'm considering is the thing I should have considered going into college, "figure out the best way to covert math aptitude into dollars and donate what you don't need to charity." One sub-option is to look into computer programming as suggested in HPMOR author's note a month or two ago. My dad thinks I should take some more stats and go for work as an analyst for some big eastern firm. And there are very likely options in this area that I'm missing.
~~~~~
I think that covers most of the relevant information I have. Now, what am I missing?
I speak as someone who is about to enter their first year of grad school in AI; note that I have also already spent a year doing robotics and a year doing AI research, so I am slightly less naieve than the typical first-year grad student.
The first half of this comment will discuss my thoughts on the merits of AI / AGI / FAI research. The second half will discuss how to do AI research if you want to. They will be separated by a bunch of dashes (-------).
I personally chose to go into AI research for a few reasons, although I don't have a high degree of certainty that it's the right thing to do. But my reasons were --- high degree of aptitude and belief that FAI research uninformed by current state of the art in AI / ML is much more likely to fail (I also think that FAI is much more a social problem than a technical problem; but I'm not sure if that's relevant).
I am pretty skeptical of AGI projects in general, mostly based on an outside view and based on looking at a small number of them. Claims of near-term AI indicate a lack of appreciation for the difficulty of the problem (both engineering barriers that seem surmountable with enough man-hours, and theoretical barriers that require fundamentally new insights to get past). I don't want to say that it's impossible that there is some magical way around these, but to me it pattern-matches to amateur mathematicians proposing approaches to Fermat's Last Theorem. If you have some particular AGI project that you think is likely to work, I'm happy to look at it and make specific comments.
I am less skeptical of FAI research of the form done by e.g. Wei Dai, Vladimir Nesov, etc. I view it as being on the far theoretical end of what I see as the most interesting line of research within the AI community. I also think it's possible for such research to be conducted as essentially AI research, at one of the more philosophically inclined labs (most likely actually a computational cognitive science lab rather than an AI lab, such as Tom Griffiths, Noah Goodman, Josh Tenenbaum, or Todd Kemp).
Okay, so say you want to be an AI researcher. How do you go about doing this? It turns out that research is less about mathematical ability (although that is certainly helpful) and more about using your time effectively (both in terms of being able to work without constant deadlines and in terms of being able to choose high-value things to work on). There are also a ton of other important skills for research, which I don't have time to go into now.
If you want to work at a top university (which I highly recommend, if you are able to), then you should probably start by learning about the field and then doing some good research that you can point to when you apply. As an undergrad, I was able to do this by working in labs at my university, which might not be an option for you. The harder way is to read recent papers until you get a good idea, check to see if anyone else has already developed that idea, and if not, develop it yourself, write a paper, and submit it (although to get it accepted, you will probably also have to write it in the right format; most notably, introductions to papers are notoriously hard to write well). It also helps to be in contact with other good researchers, and to develop your own sense of a good research program that you can write about in your research statement when you apply. These are all also skills that are very important as a grad student, so developing them now will be helpful later.
Unfortunately, I have to go now, but if I left anything out feel free to ask about it and I can clarify.