steven0461 comments on Inferring Our Desires - Less Wrong

37 Post author: lukeprog 24 May 2011 05:33AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (45)

You are viewing a single comment's thread. Show more comments above.

Comment author: steven0461 24 May 2011 08:19:26PM *  2 points [-]

That certainly seems to be to some extent true of real humans, but the point is that even if I'm to some extent a random hodgepodge, this does not obviously create in me an impulse to consult a brain scan readout or a table of my counterfactual behaviors and then follow those at the expense of whatever my other semi-random considerations are causing me to feel is right.

Comment author: Wei_Dai 24 May 2011 08:48:32PM *  3 points [-]

this does not obviously create in me an impulse to consult a brain scan readout or a table of my counterfactual behaviors

Sure, unless one of the semi-random considerations that moves you is "Crap, my EV is not coherent. Well I don't want to lay down and wait to die, so let's just make an AI that will serve my current desires." :)

Comment author: lessdazed 08 November 2011 03:30:58PM 1 point [-]

Incoherent considerations aren't all that bad. Even if someone prefers A to B, B to C, and C to A, they'll just spend a lot of time switching rather than waiting to die. I guess that people probably prefer changing their considerations in general, so your example of a semi-random consideration is sufficient but not at all unique or uncommon.