Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Oscar_Cunningham comments on Duller blackmail definitions - Less Wrong

7 Post author: Stuart_Armstrong 15 July 2013 10:08AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (44)

You are viewing a single comment's thread.

Comment author: Oscar_Cunningham 15 July 2013 09:47:32PM *  14 points [-]

I think the problem is just equivalent to dividing the gains from trade. We'll look at the second example first, since it is a canonical example of a trade.

I have the MacGuffin which I value at £9. You value it at £1000. Suppose that I give you the MacGuffin, you give me £9 and you put £991 on the table between us. Then we both have the utility we had when we started and we have £991 (the gains from the trade) to divide between us. Yay!

Now look at the first example. This is a canonical blackmail, which I'm going to re-write to make it look as much like the above example as possible.

I have some letters which incriminate us both. I value the letters not getting out at £M, and you value the letters not getting out at £N. Suppose I destroy the letters (rather than releasing them) and put £M on the table between us, while you put £N on the table between us. Then we both have the utility we would have had if the letters had been released but along with this we have £M+N (the "gains from the trade") to divide between us. Yay?

I think the reason why this example seems more blackmailesque is because there is a natural Schelling point for the division of the gains, namely (£M, £N), which corresponds to me destroying the letters without asking for anything. So asking you for money is rude, because I'm greedily going past the Schelling point.

The whole point of the exercise was to create a decision theory that doesn't comply with blackmail. Well, in a division of gains problem a sensible heuristic is to always demand a fair share (where "fair share" vaugely refers to some sort of Schelling point or something, depending on the problem). In the blackmail example the Schelling point is (£M, £N) so the heuristic tells us to demand at least £M, which is exactly refusing the blackmail!

Comment author: SilasBarta 16 July 2013 07:48:05AM 0 points [-]

Very well said! I would only add that your point generalizes: the differences between the two cases is the extent to which it has implications for future interaction ("moving the Schelling point"): blackmail-like situations are those where we intuit an unfavorable movement of the point (per the blackmailed) while we generally don't have such intuitions in he case of trade.