TimS comments on Consolidated Nature of Morality Thread - Less Wrong

13 Post author: Eliezer_Yudkowsky 15 April 2007 11:00PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (68)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: TimS 26 March 2012 01:14:46PM 0 points [-]

Imagine a simple decision game: Should I eat the poisonous fruit: Yes (-100), No (0). Obviously, No is the superior answer, and it didn't take publication of this decision theory result for humans to realize this. Making the decision game is writing the expected payouts of the environment - not setting them.

To take your example, as long as increasing the power of the tribe provides benefits to you (and I agree that it usually will), then reducing inter-tribe squabbling is the better long-term choice. Decision theory doesn't disagree, but isn't necessary for the conclusion. However, the incentive is already there, so there's no reason why evolution would select for a "baked-in" preference.

The fact that the environment rewards certain choices is a sufficient reason for those choices to be favored. I referenced decision theory only to have a way to rigorously identify which choices are favor by pre-existing reward structures.