Will_Newsome comments on SotW: Check Consequentialism - Less Wrong

38 Post author: Eliezer_Yudkowsky 29 March 2012 01:35AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (311)

You are viewing a single comment's thread. Show more comments above.

Comment author: Will_Newsome 24 March 2012 12:19:05PM *  6 points [-]

Identity shouldn't act as a normative consideration for an angel, maybe. For a human, "identity" is a pragmatic reification of cached complexes of moral conclusions that aren't immediately accessible for individual analysis. "Normative" is a misleading word here.

Comment author: Vladimir_Nesov 24 March 2012 01:01:20PM 0 points [-]

Identity shouldn't act as a normative consideration for an angel, maybe.

Still shouldn't for a human, even if does. It's a normative consideration, not a descriptive one.

Comment author: TheOtherDave 24 March 2012 02:28:06PM *  1 point [-]

I'm not sure this is responsive to Will's point... at least, it seems plausible that the moral considerations he considers identity to imperfectly encapsulate are also normative, which is why he refers to them as moral in the first place. That is, I think he means to challenge the idea that identity shouldn't be/isn't a normative consideration.

Comment author: Will_Newsome 24 March 2012 01:33:28PM 5 points [-]

...Is there a word for "normative given bounded rationality"?

Comment author: Vaniver 24 March 2012 05:12:26PM 3 points [-]

Prescriptive.

Comment author: Vladimir_Nesov 24 March 2012 01:39:12PM 0 points [-]

Bounded rationality is like the mass of the Sun, difficulty of the problem, not a kind of goal.

Comment author: Will_Newsome 24 March 2012 02:25:25PM *  7 points [-]

I don't understand.

If you're trying to dam a river, and you only have 100,000 bricks, then there is a normative solution, i.e., the solution that has the greatest chance of successfully damming the river. Talking about solutions that require one million bricks is talking about a different problem that is only relevant to people with millions of bricks. So when you say, "identity shouldn't act as a normative consideration", that sounds to me like, "you should already have one million bricks, there is no normative solution if you only have 100,000 bricks". Using 100,000 bricks to dam a river isn't using an approximation of the solution you would use if you had a million bricks. That's why I say "normative" is a misleading word here. It implies that you should try to approximate the million-brick solution even when you know you don't have enough bricks to do that: a tenth of a great million-brick dam is one millionth as useful as a complete 100,000-brick dam. Why not just renormalize such that your constraints are part of your environment and thus part of the problem, and find a normative solution given your constraints? Otherwise the normative solution is always to have already solved the problem. "What would Jesus do? Jesus would have had the foresight not to get into this situation in the first place." "Normative" is always relative to some set of constraints, so I don't see why normative-given-boundedness isn't a useful concept. I'm reminded of Nick Tarleton's intuition that decision theory needs to at some point start taking boundedness into account.

Comment author: Vladimir_Nesov 24 March 2012 02:36:28PM *  1 point [-]

It's useful to take the limitations of decision-making setup into account, but that is not fundamentally different from taking the number of bricks into account. The idealized criteria for comparing the desirability of alternatives don't normally depend on which alternatives are available. People shouldn't die even if it's impossible to keep them from dying.