JGWeissman comments on The Blackmail Equation - Less Wrong

13 Post author: Stuart_Armstrong 10 March 2010 02:46PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (87)

You are viewing a single comment's thread. Show more comments above.

Comment author: Stuart_Armstrong 10 March 2010 09:01:23PM *  1 point [-]

I haven't missunderstood the points - though I have, I fear, over-simplified the presentation for illustrative purposes. The key missing ingredient is that when I wrote that:

The baron must model the countess as seeing his decision as a fixed fact over which she has no influence,

implicit in that was the assumption that the baron was rational, knew his source and the countess' and would arrive at a decision in finite time - hence he must be correct in his assumption. I nearly wrote it that way, but thought this layout would be more intuitive.

It is just that a Countess designed a particular way can be blackmailed.

Indeed. Those are conditions that allow the countess to be blackmailed.

Could the countess plausibly raise herself to a superior epistemic vantage over the baron, and get out from under his thumb? Alas no.

If the countess is already in an inferior epistemic vantage point, she can't raise herself deterministically to a higher one - for instance, she cannot stop treating the baron's actions as a fixed fact, as an entity capable of doing that is not genuinly treating them as fixed already.

The rest of that section was a rather poorly phrased way of saying that two entities cannot be in superior epistemic vantage over each other.

Comment author: JGWeissman 10 March 2010 10:07:01PM 0 points [-]

The fifth fact is a consequence of the previous ones.

It seems that by "consequence" you mean "logical consequence", that is if I, observing this scenario, note that the first 5 conditions hold, I can derive that the 6th condition holds as well.

There is another interpretation though, that you mean a "causal consequence", that the baron, by having a certain model of the countess, makes that model correct, because the baron is rational and therefor will produce a correct model. What this interpretation tells us is wrong. (Eliezer, were you interpreting it this way when you said Stuart misunderstood your point?)

Comment author: Stuart_Armstrong 10 March 2010 10:37:41PM *  0 points [-]

Yes, I'm eliding Godelian arguments there... Consequences of anyone being rational and believing X have been removed.

Interestingly, in the model I produced down below, both the countess and the baron produce correct models of each other. Furthermore, the countess knows she produces a correct model of the baron (as she runs his source successfuly).

It also happens that the baron can check he has the correct model of the countess, after making his decision, by running her code. Since the countess will stop running his own code as soon as she also knows his outcome, he can know that his model was accurate in finite time.