Sewing-Machine comments on What can you do with an Unfriendly AI? - Less Wrong

16 Post author: paulfchristiano 20 December 2010 08:28PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (127)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 21 December 2010 02:24:42AM 1 point [-]

"wide" prior over the computational multiverse outside its box

Can you explain what you mean?

Comment author: cousin_it 21 December 2010 10:18:57AM *  0 points [-]

If the AI uses something like the Solomonoff prior, it can work out which worlds are most likely to contain such an AI. With a little intelligence it can probably figure out from its programming that humans are bipedal, that we run many AIs in boxes, that aliens on Alpha Centauri have built another AI that can turn out really helpful, etc.

Comment author: [deleted] 21 December 2010 01:56:43PM 1 point [-]

An AI with an uncomputable prior and an infinite amount of memory and time might be able to learn those things from its source code. But I think this is a bad way to approximate what a real superintelligence will be capable of.