You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Vladimir_Nesov comments on [Altruist Support] How to determine your utility function - Less Wrong Discussion

7 Post author: Giles 01 May 2011 06:33AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (45)

You are viewing a single comment's thread.

Comment author: Vladimir_Nesov 01 May 2011 09:31:42AM *  1 point [-]

In other words, the wrong way to determine your utility function is to think about what decisions you have made, or feel that you would make, in different situations.

Not quite. Examining in detail the actual reasons for your decisions made in the past or those you could've counterfactually made can show you which real-world factor or cognitive mechanism should've been different how, which would allow you to obtain a specific better judgment in that situation (or a more general class of situations). This is the way you can point out cognitive biases. It's wrong to discard your existing intuitions, even if you know that they are no good in the greater scheme of things, because they may well be the best you've got, or the best raw material you've got for developing better ones.

Comment author: Giles 01 May 2011 03:14:07PM 1 point [-]

Agreed - I was too dismissive of what can be learnt from past decision-making experiences.

I was just pointing out that "I'm always timid in social situations therefore I want to be timid in social situations" is invalid reasoning.

Comment author: Vladimir_Nesov 01 May 2011 03:24:11PM *  2 points [-]

"I'm always timid in social situations therefore I want to be timid in social situations" is invalid reasoning.

To a first approximation, and as a heuristic rule, it is valid. There are specific additional reasons to believe the conclusion invalid, and they have to do with things other than the way the initial faulty conclusion was generated. You believe the reasoning invalid because you know the conclusion to be invalid, but not the other way around.

Comment author: Giles 01 May 2011 03:45:05PM 1 point [-]

I meant "want" as in "this is one of my life goals; I would not wish to self-modify to be any other way"