mstevens comments on Harry Potter and the Methods of Rationality discussion thread, part 10 - Less Wrong

11 Post author: Oscar_Cunningham 07 March 2012 04:46PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (641)

You are viewing a single comment's thread. Show more comments above.

Comment author: KPier 13 March 2012 12:32:51AM *  8 points [-]

It's also mentioned in Circular Altruism.

This matches research showing that there are "sacred values", like human lives, and "unsacred values", like money. When you try to trade off a sacred value against an unsacred value, subjects express great indignation (sometimes they want to punish the person who made the suggestion).

My favorite anecdote along these lines - though my books are packed at the moment, so no citation for now - comes from a team of researchers who evaluated the effectiveness of a certain project, calculating the cost per life saved, and recommended to the government that the project be implemented because it was cost-effective. The governmental agency rejected the report because, they said, you couldn't put a dollar value on human life. After rejecting the report, the agency decided not to implement the measure.

Trading off a sacred value (like refraining from torture) against an unsacred value (like dust specks) feels really awful. To merely multiply utilities would be too cold-blooded - it would be following rationality off a cliff...

I'm sure there's a hint in there, but I don't know what it is.

Comment author: mstevens 13 March 2012 12:26:38PM 2 points [-]

I didn't spot that.

Probably a better source than mine, as it reflects EY's thoughts on things.