You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Dorikka comments on A Thought on Pascal's Mugging - Less Wrong Discussion

12 Post author: komponisto 10 December 2010 06:08AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (159)

Comments (159)

You are viewing a single comment's thread.

Comment author: Dorikka 01 April 2011 01:48:11AM 1 point [-]

I think that the more general problem is that if the absolute value of the utility that you attach to a world-state increases faster than does its complexity decreases given the current situation then the very possibility of that world-state existing will cause it to hijack the entirety of your utility function (assuming that there are no other world-states in your utility function which go FOOM in a similar fashion.)

Of course, utility functions are not constructed to avoid this problem, so I think that it's incredibly likely that each unbounded utility function has at least one world-state which would render it hijackable in such a manner.

Comment author: komponisto 01 April 2011 05:25:57AM *  1 point [-]

I think that the more general problem is that if the absolute value of the utility that you attach to a world-state increases faster than does its complexity decreases given the current situation then the very possibility of that world-state existing will cause it to hijack the entirety of your utility function

Yes, that's exactly the problem.

Of course, utility functions are not constructed to avoid this problem

Well, they had better be, or they will fall victim to it.

You have to choose one of the following: (1) Pascal's Mugging; (2) Scope Insensitivity (bounding utility by improbability); or (3) Wishful Thinking (bounding improbability by utility).

Comment author: wedrifid 01 April 2011 05:37:53AM *  4 points [-]

I think that the more general problem is that if the absolute value of the utility that you attach to a world-state increases faster than does its complexity decreases given the current situation then the very possibility of that world-state existing will cause it to hijack the entirety of your utility function

Yes, that's exactly the problem.

We often call such things a 'problem' yet by very definition it is exactly how it should be. If your utility function genuinely represents your preferences (including preferences with respect to risk) then rejoice in the opportunity to devote all your resources to the possibility in question! If it doesn't then the only 'problem' is that your 'utility function', well, isn't your actual utility function. It's the same problem that you get when you think you like carrots when you really like peaches.

Voluntary dedication is not 'hijacking'.

(Response primarily directed to quoted text and only a response to the parent in as much as it follows the problem frame.)

Comment author: komponisto 01 April 2011 05:41:58AM 1 point [-]

Agreed.

Comment author: XiXiDu 01 April 2011 01:19:31PM 0 points [-]

If it doesn't then the only 'problem' is that your 'utility function', well, isn't your actual utility function. It's the same problem that you get when you think you like carrots when you really like peaches.

Our heuristics hijack our volition?

Powered by Reddit Powered by Reddit