In the past, people like Eliezer Yudkowsky (see 1, 2, 3, 4, and 5) have argued that MIRI has a medium probability of success. What is this probability estimate based on and how is success defined?
I've read standard MIRI literature (like "Evidence and Import" and "Five Theses"), but I may have missed something.
-
(Meta: I don't think this deserves a discussion thread, but I posted this on the open thread and no-one responded, and I think it's important enough to merit a response.)
I agree with Eliezer that the main difficulty is in getting top-quality, relatively rational people to spend hundreds of hours being educated, working through the arguments, etc.
Jaan has done a surprising amount of that and also read most or all of the Sequences. Thiel has not yet decided to put in that kind of time.
Here's a list of people I'd want on that committee if they were willing to put in hundreds of hours catching up and working through the arguments with us: Scott Aaronson, Peter Norvig, Stuart Russell, Michael Nielsen.
I'd probably be able to add lots more names to that list if I could afford to spend more time becoming familiar with the epistemic standards and philosophical sophistication of more high-status CS people. I would trust Carl Shulman, Paul Christiano, Jacob Steinhardt, and a short list of others to add to my list with relatively little personal double-checking from me.
But yeah; the main problem seems to me that I don't know how to get 400 hours of Andrew Ng's time.
Although with Ng in particular it might not take 400 hours. When Louie and I met with him in Nov. '12 he seemed to think AI was almost certainly a century or more away, but by May '13 (after getting to do his deep learning work on Google's massive server clusters for a few months) he changed his tune, saying "It gives me hope –- no, more than hope –- that we might be able to [build AGI]... We clearly don’t have the right algorithms yet. It’s going to take decades. This is not going to be an easy one, but I think there’s hope.” (On the other hand, maybe he just made himself sound more optimistic than he anticipates inside because he was giving a public interview on behalf of pro-AI Google.)
This is a great answer but actually a little tangential to my question, sorry for being vague. Mine was actually about the part of shminux's proposal that involved finding potential mega donors. Relatedly, how much convincing do you think it would take to get Tallinn or thiel to increase their donations by an order of magnitude, something they could easily afford? This seems like a relatively high leverage plan if you can swing it. With x million dollars you can afford to actually pay to hire people like google can, if on a much smaller scale.