You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

wedrifid comments on Less Wrong: Open Thread, December 2010 - Less Wrong Discussion

7 Post author: David_Gerard 06 December 2010 02:29PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (84)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 11 December 2010 09:27:15PM 5 points [-]

Michael Vassar is usually the voice within SIAI of such concerns. It hasn't been formally written up yet, but besides the Last Judge notion expressed in the original CEV paper, I've also been looking favorably on the notion of giving a binary veto over the whole process, though not detailed control, to a coherent extrapolated superposition of SIAI donors weighted by percentage of income donated (not donations) or some other measure of effort exerted.

And before anyone points it out, yes I realize that this would require a further amendment to the main CEV extrapolation process so that it didn't deliberately try to sneak just over the veto barrier.

Look, people who are carrying the Idiot Ball just don't successfully build AIs that match up to their intentions in the first place. If you think I'm an idiot, worry about me being the first idiot to cross the Idiot Finish Line and fulfill the human species' destiny of instant death, don't worry about my plans going right enough to go wrong in complicated ways.

Comment author: wedrifid 12 December 2010 02:33:42AM 1 point [-]

Look, people who are carrying the Idiot Ball just don't successfully build AIs that match up to their intentions in the first place

Woah there. I remind you that what prompted your first reply here was me supporting you on this particular subject!