I'm starting my Honours next year, and would like to do something towards helping MIRI with Friendly AI. I would also prefer to avoid duplicating any of MIRI's work (either already done, or needed to be done before my honours are finished midway through 2015). I decided to post this here rather than directly email MIRI as I guessed a list of potential projects would probably be useful for others as well (in fact, I was sure such a thing had already been posted, but I was unable to find it if it did in fact exist). So: what sort of Friendly AI related projects are there that could potentially be done by one person in a year of work? (I suppose it would make sense to include PhD-length suggestions here as well).
Some notes about me and my abilities: I am reasonably good with math, though my understanding of probability, model theory and provability logic are lacking (I will have a few months before hand that I plan to use to try and learn whatever maths I will need that I don't already have). I am a competent Haskell programmer, and (besides AI) I am interested in dependent type systems, total languages, and similar methods of proving certain program errors cannot occur, although I would have to do some background research to learn more of the state of the art in that field. I would (hesitantly) guess that this would be the best avenue for something that a single person could do that might be useful, but I'm not sure how useful it would be.
Fairly technical would be good. IEM and the sociological work are somewhat outside my interests. Attending a workshop would unfortunately be problematic; anxiety issues make travelling difficult, especially air travel (I live in Australia). Writing up comments on the research papers is an excellent idea; I will certainly start doing that regardless of what project I do. Of the subjects listed, I am familiar (in roughly decreasing order) with functional programming, efficient algorithms, parallel computing, discrete math, numerical analysis, linear algebra, and the basics of set theory and mathematical logic. I have "Naive Set Theory", "Introduction to Mathematical Logic", and "Godel, Escher, Bach" sitting on my desk at the moment, and I am currently taking courses in theory of computation, and intelligent systems (a combination AI/machine learning/data mining course). The areas I had planned to learn after the above are incompleteness/undecidability, model theory, and category theory. In terms of how my prospective advisor could affect things, he's mostly interested in cognitive science based AI, with some side interest in theory of computation.
In that case, I think you'll want to study mathematical logic, theory of computation, incompleteness/undecidability and model theory, to improve your ability to contribute to the open problems that Eliezer thinks are most plausibly relevant to Friendly AI. Skimming our recent technical papers (definability of truth, robust cooperation, tiling agents) should also give you a sense of what you'd need to learn to contribute at the cutting edge.
A few years from now, I hope to have write-ups of a lot more open problems, including ones that don't rely so heavily ... (read more)