Nick_Tarleton comments on Wanted: backup plans for "seed AI turns out to be easy" - Less Wrong

18 Post author: Wei_Dai 28 September 2011 09:54PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (62)

You are viewing a single comment's thread. Show more comments above.

Comment author: Nick_Tarleton 29 September 2011 05:55:13PM 3 points [-]

Make millions of nearly identical seed AIs, each with the utility function of an individual.

You assume this is much easier than CEV. That doesn't seem terribly likely to me. (Individuals are more coherent, agent-like, and aware of their values than collectives, but still not very any of those things.)

Comment author: lessdazed 29 September 2011 09:53:05PM 2 points [-]

You assume...doesn't seem terribly likely

Sort of. I said in my post on the subject that I think it probably isn't easier, but there is a high probability that it is, and no one knows exactly how to calculate either, as far as I can tell. So as an alternative under the assumption that CEV is too hard and seed AI is relatively easy, it's a good alternative in my mind - if it's the cohering that's causing the difficulty, then it's a reasonably possible alternative.