timtyler comments on Less Wrong: Open Thread, December 2010 - Less Wrong

7 Post author: David_Gerard 06 December 2010 02:29PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (84)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 11 December 2010 09:27:15PM 5 points [-]

Michael Vassar is usually the voice within SIAI of such concerns. It hasn't been formally written up yet, but besides the Last Judge notion expressed in the original CEV paper, I've also been looking favorably on the notion of giving a binary veto over the whole process, though not detailed control, to a coherent extrapolated superposition of SIAI donors weighted by percentage of income donated (not donations) or some other measure of effort exerted.

And before anyone points it out, yes I realize that this would require a further amendment to the main CEV extrapolation process so that it didn't deliberately try to sneak just over the veto barrier.

Look, people who are carrying the Idiot Ball just don't successfully build AIs that match up to their intentions in the first place. If you think I'm an idiot, worry about me being the first idiot to cross the Idiot Finish Line and fulfill the human species' destiny of instant death, don't worry about my plans going right enough to go wrong in complicated ways.

Comment author: timtyler 11 December 2010 09:45:33PM 1 point [-]

coherent extrapolated superposition of SIAI donors

Is that what they mean by "getting the inside track on the singularity"? ;-)

Comment author: Eliezer_Yudkowsky 11 December 2010 09:59:37PM 1 point [-]

It gets to possibly say "No", once. Nothing else.

Are you under the impression that jumping on statements like this, after the original statement explicitly disclaimed them, is a positive contribution to the conversation?

Comment author: timtyler 11 December 2010 10:04:21PM *  3 points [-]

Yu'El - please don't you jump on me! I was mostly trying to be funny. Check with my smiley!

This was a reference to Jaron Lanier's comment on this topic - in discussion with you.