You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Giles comments on [Altruist Support] How to determine your utility function - Less Wrong Discussion

7 Post author: Giles 01 May 2011 06:33AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (45)

You are viewing a single comment's thread. Show more comments above.

Comment author: Giles 01 May 2011 03:40:35PM 0 points [-]

In a community where instrumental rationality is high-status, there will be social pressure to behave according to your stated utility function. So you have to be careful to at least state a function that's sufficiently compatible with your real one that people won't notice the discrepancy. If they do, they will try to help you overcome your "akrasia".

Note that I said "compatible", not "similar". So your real utility function could be "complete selfishness". If complete selfishness requires support from a rationalist community, you may wish to signal cooperation by stating that your utility function is "complete altruism".

The result of this is that you will find doing lots of world-improving stuff (to keep the support of the rationalist community) , and your true selfish utility function will ensure you have lots of fun, find lots of sexual opportunities, etc. while doing it.

I should have mentioned the "stated" vs. "private" distinction in the above post. I'll write these ideas up in a future post, but I'll probably need to explain a bit more of my view on signalling first.