I suspect the easiest path to AGI is to just throw a ton of bodies and computing power at the problem, build a Kluge AI, and let it stumble its way into recursive self-improvement. This is what Larry Page is trying to do. I don't expect it to work this time, but if China or the NSA or Google or Goldman Sachs tries to do it with the computing power and AI researchers we'll have 35 years from now, they very well might succeed, even without any deep philosophical insights. After all, this is how evolution built general intelligence: no philosophical insight, just a bunch of specialized modules kludged together, some highly general learning algorithms, and lots of computing power. The problem is that this approach is very unlikely to yield something capable of Friendliness, and yet there are massive nearer-term incentives for China and the NSA and everyone else to race towards it.

Ah, yes, you expressed better than I could my other reason for thinking AI is most likely to be built by a big organization. I'd really been struggling how to say that.

One thought I have, building on this comment of yours, is that while making kludge AI safe may look impossible, given that sometimes you have to shut up and do the impossible, I wonder if making kludge AI safe might be the less-impossible option here.

EDIT: I'm also really curious to know how Eliezer would respond to the paragraph I quoted above.