xxd comments on Wanted: backup plans for "seed AI turns out to be easy" - Less Wrong

18 Post author: Wei_Dai 28 September 2011 09:54PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (62)

You are viewing a single comment's thread. Show more comments above.

Comment author: xxd 20 December 2011 09:13:13PM 0 points [-]

Will the uploaded good humans still stay good once they are uploaded?

What is the test to make sure that a regular (though very smart) human will remain "good"?

Don't we have to select from a small pool of candidates who have not succumbed to temptation to abuse power?

I have to say that I can think of very few candidates who would pass muster in my mind and not abuse power.

Even here: Reading through many of the posts, we have widely conflicting "personal utility functions" most of whose owners would consider to be "good" utility functions and yet those who do not hold the same personal utility functions could consider the utility functions of others to be "bad".

To my way of thinking it's incredily risky to try to upload "good" humans.