orthonormal comments on The Domain of Your Utility Function - Less Wrong

24 Post author: Peter_de_Blanc 23 June 2009 04:58AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (94)

You are viewing a single comment's thread. Show more comments above.

Comment author: orthonormal 23 June 2009 11:40:08PM 0 points [-]

Well, the code for the seed should be pretty simple, at least. But I don't see how that defeats the purpose of my statement; it may be that short of enlisting a superintelligence to help, all current attempts to approximate and extrapolate human preferences in a consistent fashion (e.g. explicit ethical or political theories) might be too crude to have any chance of success (by the standard of actual human preferences) in novel scenarios. I don't believe this will be the case, but it's a possibility worth keeping an eye on.