Alicorn comments on Open Thread, December 1-15, 2012 - Less Wrong

5 Post author: OpenThreadGuy 01 December 2012 05:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (177)

You are viewing a single comment's thread. Show more comments above.

Comment author: Alicorn 10 December 2012 01:24:01AM *  2 points [-]

No problem. You can only have a preference that doesn't get traded off with if it happens to never conflict with anything - for instance, my preference that there be a moon has yet to interact with any other preferences I could act towards fulfilling; the moon just goes on being regardless - or, if it's your only preference. Even if you have only one preference, there could be tradeoffs about instrumental subgoals. You might have to decide between a 50% chance of ten units of preference-fulfillment and a guarantee of five units, even if you'd really like to have both at once, even if the units are the only thing you care about.