Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

PhilGoetz comments on Why the beliefs/values dichotomy? - Less Wrong

20 Post author: Wei_Dai 20 October 2009 04:35PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (153)

You are viewing a single comment's thread.

Comment author: PhilGoetz 22 October 2009 09:39:01PM *  5 points [-]

It seems clear that our preferences do satisfy Independence, at least approximately.

How big of a problem does this simple example signify?

  • A = I acquire a Nintendo
  • B = I acquire a Playstation
  • C = I acquire a game for the Nintendo
  • D = I acquire a game for the Playstation
  • A&C > B&C but A&D < B&D
Comment author: Wei_Dai 24 October 2009 11:29:16AM 3 points [-]

Your example shows that we can't assign utilities to events within a single world, like acquiring game systems and games, and then add them up into a utility for that world, but it's not a counterexample to Independence, because of this part:

A and B are what happens in one possible world, and C and D are what happens in another.

Independence is necessary to assign utilities to possible world histories and aggregate those utilities linearly into expected utility. Consider the apples/oranges example again. There,

  • A = I get an apple in the world where coin is heads
  • B = I get an orange in the world where coin is heads
  • C = I get an apple in the world where coin is tails
  • D = I get an orange in the world where coin is tails

Then, according to Independence, my preferences must be either

  1. A&C > B&C and A&D > B&D, or
  2. A&C < B&C and A&D < B&D

If case 1, I should pick the transparent box with the apple, and if case 2, I should pick the transparent box with the orange.

(I just realized that technically, my example is wrong, because in case 1, it's possible that A&D > A&C and B&D > B&C. Then, I should most prefer an opaque box that contains an apple if the coin is heads and an orange if the coin is tails, since that gives me outcome A&D, and least prefer an opaque box that contains the opposite (gives me B&C). So unless I introduce other assumptions, I can only derive that I shouldn't simultaneously prefer both kinds of opaque boxes to transparent boxes.)