Vladimir_Nesov comments on Formalizing Value Extrapolation - Less Wrong

14 Post author: paulfchristiano 26 April 2012 12:51AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (48)

You are viewing a single comment's thread. Show more comments above.

Comment author: Vladimir_Nesov 26 April 2012 06:09:44PM *  2 points [-]

Maybe the AI could also ask the hypothetical human upload for the right universal distribution to use?

Certainly, but it's not even clear what that distribution should be over, and whether considering probability distributions at all is the right thing to do. The initial AGI needs some operating principles, but these principles should allow fundamental correction, as the research on the inside proceeds (for which they have to be good enough initially).