The reply that it "is impossible for A and ~A to both be evidence for B" is to ignore what Frank said in favor of insisting on the very overgeneralization I think he was trying to point out. It's not impossible at all when we are being imprecise enough about the prior expectations involved, such as when we lump all moments in a sustained effort together.
Here's an example to illustrate what I'm saying: Say you are a parent of a 10 year old boy who generally wants to stay up past his bedtime. His protests vary from occasional temper tantrums to the usual slumped-shoulders expression of disappointment that bedtime has finally arrived. Under normal circumstances, the expectation is that he would give at least some evidence of wanting to stay up later. We'll call this resistance "A," and A is evidence for "B": his desire and motivation to stay up later. What shall we say when ~A happens? That is, what shall we say when the boy one day suddenly goes enthusiastically to bed? That he has given up his desire? That it is impossible for this to be evidence of his continued desire and motivation? Of course not. It is exactly what we would expect a motivated and reasonably intelligent person to do: try different and probably more effective strategies. If we generalize the ongoing experience of the little boy's quest to say up later, A and ~A are both evidence of B. "It is impossible for A and ~A to both be evidence for B" is simply not narrow enough to be a true statement, and using it in that way can easily amount to a bad counterargument.
Rather, we need to be specific about each situation. What I think we should pay attention to here is the prior expectation of B. With a high enough prior, A and ~A could either (but not both) be evidence of B. But if we are not being specific to each precise situation, the generalization "it is impossible for A and ~A to both be evidence of B" can be a very subtle straw man, because the person being argued against may not be relying on the assumption that A and ~A are equal evidence for B at the same time and in the same situation.
Returning to the Japanese Fifth Column argument, unlike the little boy in my example the Japanese (and, in general, descendants from countries that go to war with their current country of citizenship) do not have a consistent track record of wartime sabotage. Also, there isn't any reason to think they would not generally be more loyal to their country of citizenship than the country of their parents, grandparents, or even of their own childhood. So there is no particularly strong expectation that they would commit sabotage... and thus no such expectation that some mysterious lack of sabotage is itself a sign of a new strategic attempt as part of a sustained effort. The argument should come down to the prior expectation of Japanese sabotage. That seems to be the crux of it to me.
It seems to me the weakness in Frank's argument also lies in the basic premise that we should expect the Japanese to commit sabotage. And I believe the governor would need to rely on that premise, or a similar one, in order to sustain his argument beyond what Eliezer presented.
But. The inside information premise seems nearly undefeatable to me. We can't comment on information we don't have. I think that is always a possibility with controversial official responses that most people would prefer to deny. If the person whose claims you are evaluating has secret but pertinent information you don't have access to, then it can be very difficult to offer a fair analysis. For one, you will have very different yet subjectively valid prior expectations.
If we generalize the ongoing experience of the little boy's quest to say up later, A and ~A are both evidence of B.
You seem to be using "evidence of X" to mean something along the lines of "consistent with X". That's not what it means in this context.
An event is evidence for or against a scenario insofar as it changes your subjective probability estimate for that scenario. Your example child going enthusiastically to bed is in fact evidence that he's changed his mind about staying up past his bedtime: it makes that scenario subje...
From Robyn Dawes’s Rational Choice in an Uncertain World:
Consider Warren’s argument from a Bayesian perspective. When we see evidence, hypotheses that assigned a higher likelihood to that evidence gain probability, at the expense of hypotheses that assigned a lower likelihood to the evidence. This is a phenomenon of relative likelihoods and relative probabilities. You can assign a high likelihood to the evidence and still lose probability mass to some other hypothesis, if that other hypothesis assigns a likelihood that is even higher.
Warren seems to be arguing that, given that we see no sabotage, this confirms that a Fifth Column exists. You could argue that a Fifth Column might delay its sabotage. But the likelihood is still higher that the absence of a Fifth Column would perform an absence of sabotage.
Let E stand for the observation of sabotage, and ¬E for the observation of no sabotage. The symbol H1 stands for the hypothesis of a Japanese-American Fifth Column, and H2 for the hypothesis that no Fifth Column exists. The conditional probability P(E | H), or “E given H,” is how confidently we’d expect to see the evidence E if we assumed the hypothesis H were true.
Whatever the likelihood that a Fifth Column would do no sabotage, the probability P(¬E | H1), it won’t be as large as the likelihood that there’s no sabotage given that there’s no Fifth Column, the probability P(¬E | H2). So observing a lack of sabotage increases the probability that no Fifth Column exists.
A lack of sabotage doesn’t prove that no Fifth Column exists. Absence of proof is not proof of absence. In logic, (A ⇒ B), read “A implies B,” is not equivalent to (¬A ⇒ ¬B), read “not-A implies not-B .”
But in probability theory, absence of evidence is always evidence of absence. If E is a binary event and P(H | E) > P(H), i.e., seeing E increases the probability of H, then P(H | ¬ E) < P(H), i.e., failure to observe E decreases the probability of H . The probability P(H) is a weighted mix of P(H | E) and P(H | ¬ E), and necessarily lies between the two.1
Under the vast majority of real-life circumstances, a cause may not reliably produce signs of itself, but the absence of the cause is even less likely to produce the signs. The absence of an observation may be strong evidence of absence or very weak evidence of absence, depending on how likely the cause is to produce the observation. The absence of an observation that is only weakly permitted (even if the alternative hypothesis does not allow it at all) is very weak evidence of absence (though it is evidence nonetheless). This is the fallacy of “gaps in the fossil record”—fossils form only rarely; it is futile to trumpet the absence of a weakly permitted observation when many strong positive observations have already been recorded. But if there are no positive observations at all, it is time to worry; hence the Fermi Paradox.
Your strength as a rationalist is your ability to be more confused by fiction than by reality; if you are equally good at explaining any outcome you have zero knowledge. The strength of a model is not what it can explain, but what it can’t, for only prohibitions constrain anticipation. If you don’t notice when your model makes the evidence unlikely, you might as well have no model, and also you might as well have no evidence; no brain and no eyes.
1 If any of this sounds at all confusing, see my discussion of Bayesian updating toward the end of The Machine in the Ghost, the third volume of Rationality: From AI to Zombies.