Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Will_Sawin comments on Pancritical Rationalism Can Apply to Preferences and Behavior - Less Wrong

1 Post author: TimFreeman 25 May 2011 12:06PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (29)

You are viewing a single comment's thread. Show more comments above.

Comment author: Will_Sawin 26 May 2011 05:52:39PM 1 point [-]

I agree, if we could start the process with the subject's true preferences, and the subject were rational. Instead it seems we have to start with the results from introspection, which might be wrong. I'm trying to understand what to do about that. I think people should take the possibility of incorrect introspection seriously.

Then you're just dealing with beliefs about preferences, which are a kind of beliefs, so this reduces to PCR for beliefs.

Comment author: TimFreeman 26 May 2011 08:06:55PM *  2 points [-]

Then you're just dealing with beliefs about preferences, which are a kind of beliefs, so this reduces to PCR for beliefs.

You're right there. And PCR for beliefs is trivial in principle, just use Bayes' rule and the Universal Prior based on the programming language of your choice. Nobody seems to be good enough at actually evaluating that prior to care much about which programming language you use to represent the hypotheses yet.

So if someone introspects and says they will make choices as though they have unbounded utility, and the math makes it seem impossible for them to really do that, then I can reply "I don't believe you" and move on, just as though they had professed believing in an invisible dragon in their garage.

That's a really simple solution to get rid of a large pile of garbage, contingent on the math working out right. Thanks. I'll pay more attention to the math.

ETA: I edited the OP to point to this comment. This was an excellent outcome from the conversation, by the way. LessWrong works.

Comment author: Will_Sawin 26 May 2011 10:12:51PM 0 points [-]

(obligatory xkcd reference)

LessWrong: It works, bitches.