bigjeff5 comments on Absence of Evidence Is Evidence of Absence - Less Wrong

54 Post author: Eliezer_Yudkowsky 12 August 2007 08:34PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (108)

Tags: bayesian

Comments (108)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: Tyrrell_McAllister 30 January 2011 01:14:24AM * 9 points [-]

Warren stated in the quote that the lack of any subversive activity was the most convincing factor of all the evidence he has that the 5th Column would soon commit subversive activity.

(Emphasis added.)

I just don't see that in the quote. Here is the Warren quote from the OP:

"I take the view that this lack [of subversive activity] is the most ominous sign in our whole situation. It convinces me more than perhaps any other factor that the sabotage we are to get, the Fifth Column activities are to get, are timed just like Pearl Harbor was timed... I believe we are just being lulled into a false sense of security."

His claim isn't that subversive activity will start soon. The claim is that subversive activity will be "timed just like Pearl Harbor was timed". I read this to mean that he anticipates a centrally-orchestrated, synchronized, large-scale attack, of the sort that could only be pulled off by a disciplined, highly-competent fifth column.

If they had seen small, piece-meal efforts at sabotage, then that would have been evidence against a competent fifth column. That is, P(there is a competent fifth column | there has been piece-meal sabotage) < P(there is a competent fifth column).

Therefore, not seeing such efforts is evidence for a competent fifth column: P(there is a competent fifth column | there has been no piece-meal sabotage) > P(there is a competent fifth column). This is a direct algebraic consequence of Bayes's formula.

Of course, seeing no piece-meal sabotage is also evidence for there being no fifth column at all. But if your prior for "no fifth column" is sufficiently low, it still makes sense to spend most of your effort on interpreting what the no-sabotage evidence says about the nature of the fifth column, given that it exists. And what it says, given that there is a fifth column, is that the fifth column is probably marshaling its forces to strike a major blow. (Or at least, that's what the no-sabotage evidence says under the right prior.)

As soon as any subversive activity occurs, the evidence that the 5th Column is going to commit subversive activity clearly just went down!

Scattered and piecemeal acts of sabotage would show that the fifth column is incompetent. So such activity would make "our situation" less "ominous". This is consistent with Warren's view. Such sabotage wouldn't make the probability of subversive activity go down, but Warren doesn't say that it would. But such sabotage would make the probability of sabotage comparable to Pearl Harbor go down. That is Warren's claim.

And since the lack of evidence was the strongest evidence for this fact, the fact that the "lack of evidence" is now 0 (either evidence exists, or no evidence exists, there are no degrees for this type of evidence) ...

This is where we disagree. It's not a matter of "sabotage" vs. "no sabotage". Incompetent sabotage is different from competent sabotage. Warren has a prior that assigns a high prior probability to the existence of a fifth column. His priors about how fifth-columns work, as a function of their competence, are evidently such that our significantly-probable states, in increasing order of ominousness, are

having seen incompetent sabotage,
having seen no sabotage yet,
having seen competent sabotage.

Warren believed that we were in the middle state.

The absurdity of this reasoning should be obvious, and it should be thrown out immediately. The lack of subversive activity was clearly not evidence that the 5th Column was planning something. It could not be. You might think the 5th Column was planning something based on other evidence, and that is perfectly fine, but your reasoning for the risk of a subversive activity cannot be based on the lack of any subversive activity. It must be based on other evidence or it invalidates itself.

In my previous comment, I gave a Bayesian explanation of how the lack of subversive activity could be evidence that we are in a more dangerous situation than we would have been if we had seen evidence of subversion. That is, given the right priors, the lack of subversive activity could be "ominous". Can you point to an error in my reasoning?

Comment author: bigjeff5 comment score below threshold [+] (15 children)

Comment author: Tyrrell_McAllister 30 January 2011 05:14:32AM * 3 points [-]

Of course, seeing no piece-meal sabotage is also evidence for there being no fifth column at all.

That makes no sense at all. How can a fact be both for and against the same thing? You can't split the evidence. You can't say 20% of the time it means there isn't a 5th Column and 80% of the time it means there is.

It is possible to have two mutually exclusive propositions, Q and R, such that some observation E is both evidence for Q and evidence for R. That is, it is possible to have both p(Q|E) > p(Q) and p(R|E) > p(R), even though Q implies not-R.

Do you disagree?

In my argument above,

Q is "there is a competent fifth column",
R is "there is no fifth column", and
E is "there has been no piece-meal sabotage".

Before moving on to the other issues, does this part make sense now?

ETA: It's important to note that Q and R, while mutually exclusive, are not exhaustive. A third possibility is that there is fifth column, but it's incompetent. Q is not equivalent to not-R.

Comment author: bigjeff5 30 January 2011 07:57:59PM 0 points [-]

Warren did not say anything about piece-meal sabotage. He called it a "lack of subversive activity", and he didn't put limits on what type of subversion.

The way you described E is also counter-intuitive and confusing, and is not the way Warren described it. E must be that there has been subversive activity, that's what Warren described. He said the lack of E (~E) was evidence of Q (p(Q|~E)). That's fine, as long as E must be evidence that Q is not true (p(~Q|E)).

Since ~E is weak evidence for Q, E is strong evidence for ~Q. The next subversive activity of any kind will be the first instance of E, and it will invalidate his entire reasoning for detaining the Japanese-American citizens.

Comment author: Tyrrell_McAllister 30 January 2011 09:06:53PM * 2 points [-]

Warren did not say anything about piece-meal sabotage. He called it a "lack of subversive activity", and he didn't put limits on what type of subversion.

The way you described E is also counter-intuitive and confusing, and is not the way Warren described it.

Okay, replace my earlier definition of E with

E is "we have seen no subversive activity".

Do you agree that, under some priors, you could have p(Q|E) > p(Q) and p(R|E) > p(R), even though Q implies not-R?

Set aside the question of whether these are reasonable priors. My point was only this; Warren didn't make the simple mistake with the probability calculus that Eliezer thought Warren made. He wasn't simultaneously asserting p(H|E) > p(H) and p(H|~E) > p(H). That would be wrong under any prior, no matter how bizarre. But it's not what Warren was doing.

What Warren said is consistent with coherent Bayesian updating, even if he was updating on a bizarre prior. It might have been wrong to put a high prior probability on subversive activity, but the probability calculus doesn't tell you how to pick your prior. All I am saying is that the Warren quote, in and of itself, does not constitute a violation of the rules of the probability calculus.

Maybe Warren committed such a violation earlier on. Maybe that's how he arrived at such a high prior for the existence of subversive activity. But those earlier steps in his reasoning aren't laid out before us here, so we can't point to any specific misapplication of Bayes's rule, as Eliezer tried to do.

Comment author: bigjeff5 31 January 2011 12:55:22AM * 0 points [-]

E is "we have seen no subversive activity".

I don't like the way you describe that. It is confusing. The evidence is subversive activity. You cannot go out and look for no subversive activity, that makes no sense. You have to look for subversive activity. I'm not sure why you're fighting so hard for this point, since not finding something suggests just as much as finding something does. The only reason I suggest a change is for clarity. I don't want to think about no subversive activity and not no subversive activity, I want to think about subversive activity or no subversive activity. There is no difference, the second is simply less confusing.

E is subversive activity, and Warren's position is p(Q|~E).

Do you agree that, under some priors, you could have p(Q|E) > p(Q) and p(R|E) > p(R), even though Q implies not-R?

Absolutely. I said so two posts up. The question is not about Q and R, though, it's about Q and ~Q.

Set aside the question of whether these are reasonable priors.

But the whole argument is about the priors. The reason Warren's position is nonsensical is not because he believes a lack of subversion suggests some fact, it's that he argues that a lack of subversion suggests a fact, and then behaves in a manner counter to his argument. I've been arguing the fact that Warren argues p(Q|~E), But the reason he is locking up the Japanese-Americans is because he expects p(E|Q). The only way p(E|Q) makes any sense is if p(Q|E) is also true.

Warren's fundamental fear is based on p(E|Q) - that is, the 5th Column is plotting and scheming, and this will lead to subversion. The argument he uses to support this, however, is that p(Q|~E). The two positions are inversely related. If p(E|Q) is strong, then p(Q|~E) must be weak.

In other words, if p(E|Q) is strong, and p(Q) is high, then p(E) should be very high (because Q implies E), and p(~E) should be very small. Yet a very high ~E is used as evidence of p(Q). That makes no sense. If p(E|Q) is high, then ~E can exist in spite of Q, but it cannot exist because of Q.

The only way this is at all tenable is if p(E|Q) and p(~E|Q) are both true. In which case, neither E nor ~E is evidence of Q.

That's the whole point.

probability calculus doesn't tell you how to pick your prior.

The whole point of this discussion is that his reasoning does not coincide with his actions. Thus one or the other is wrong.

Comment author: CuSithBell 31 January 2011 04:09:00AM 2 points [-]

Doesn't his position make sense if he believes that:

if there's no organized fifth column, we should see some intermittent, disorganized sabotage, and
if there is an organized fifth column, we should see NO sabotage before some date, at which there is a devastating attack

Of course, I agree that it's likely he would have made a different argument if he had seen evidence of sabotage - but as presented it seems his position is at least potentially coherent.

Comment author: bigjeff5 31 January 2011 05:32:39PM -1 points [-]

But what is the date? Is it 2 months? 6 months? A year? 5 years? What if it never happens? If nothing happens in 200 years, does that mean we must be absolutely certain that a fifth column is planning an attack?

It's not evidence, it's a lack of evidence. That's the point, and that's the problem.

Warren states it is his most ominous evidence that they are planning something. What evidence? It's not there, it doesn't exist. His whole position is based on the idea that the lack of evidence indicates they are planning something, yet he has nothing to suggest that such a lack of evidence indicates anything. The only thing the fact that they haven't attacked yet is evidence for is that they haven't attacked yet. Nothing more, nothing less, unless you have a pattern of behavior to base that on. There was no such pattern for the fifth column.

Comment author: CuSithBell 31 January 2011 06:21:44PM 0 points [-]

he has nothing to suggest that such a lack of evidence indicates anything.

That's a different discussion. As you said,

The whole point of this discussion is that his reasoning does not coincide with his actions.

I was simply arguing that your characterization of his argument as inherently self-contradictory was incorrect. Yeah, his supposed priors are probably wrong, but that's a different issue.

But what is the date? Is it 2 months? 6 months? A year? 5 years?

Okay, say it's 6 months. Does that make his argument non-contradictory?

If I predict it's going to rain soon because of a long dry spell, when it rains that doesn't prove me wrong.

Comment author: bigjeff5 31 January 2011 09:08:47PM * -1 points [-]

If I predict it's going to rain soon because of a long dry spell, when it rains that doesn't prove me wrong.

Of course not, you have a pattern of weather to base that on, in which dry spells were consistently followed by rain.

Where is the basis for a lack of subversion? Historically, a lack of subversion has meant no subversion was ever planned, on what basis is this different for the 5th Column?

Okay, say it's 6 months. Does that make his argument non-contradictory?

Yes, because now your evidence is that, if there is a 5th column, major subversion occurs every 6 months. This is testable.

His classification of a lack of subversion as evidence that the 5th Column is planning a major strike flies in the face of history - he has a small handful of anomalies to rely on. That's all.

I'll point to Eliezer's example of mammograms in his "Intuitive Explanation of Bayes Theorem" to help describe what I mean, particularly since it's pretty easy to find a very in-depth beysian analysis of this particular problem by Eliezer himself. In the example, 1% of women get breast cancer. 80% of the time a mammogram will test positive if a woman has breast cancer, 20% of the time it will test negative. 10% of the time a mammogram will test positive for someone who doesn't have breast cancer. This works out to a 7.8% likelihood that a woman has cancer if she gets a positive result on a mammogram. Conversely, getting a negative result on a mammogram results in a 0.22% likelihood that a woman has cancer.

In the Warren scenario, the 5th Column planning an attack is like the 1% breast cancer rate, and finding evidence of subversion is the mammogram. Not finding any evidence of subversion is the exact same as getting a negative on a mammogram in the breast cancer scenario. It has happened, sure, but it is extremely rare and in the vast majority of cases no subversion means no planned subversion. The problem is you don't have a history of major subversion without evidence of subversion. Throughout history it has been the exact opposite, therefore a lack of subversion must have a very low probability for preceding a major subversive attack.

Warren's position is like saying he believes there is a high risk of breast cancer because the mammogram came up negative. The only reasonable response to that is WTF? Yes, it's possible that the fifth column is planning something, but you cannot assume that because the evidence says otherwise, that's not reasonable at all. You can come to the conclusion through other evidence, but not with that evidence.

What Warren managed to do is take evidence that did not support his fear and claim that it did. It doesn't make any sense, it is an unreasonable position to take.

Now, if Warren had said "There is a very low likelihood that the 5th Column is planning a surprise attack, but I am not willing to take that risk" then it's an entirely different situation, and that is a completely reasonable response. If breast cancer means being forced to fight through Dante's 9 levels of hell, then it might be worth a double-mastectomy in spite of the 1 in 500 chance that it would happen.

I was wrong when I said that a single case of subversion falsifies his position. Obviously surprise attacks exist, so that was clearly incorrect, and I think it led to a lot of the disagreement in the discussion. I was looking at the problem too narrowly. However the reason surprise attacks are a surprise is because they are very rare, so the fact that nothing has happened must still overwhelmingly support the idea that nothing will happen. In other words, it is overwhelming evidence against an attack, not for it. That's the only reason surprise attacks work at all, because you you have no evidence to suggest they are coming (and that they haven't attacked is not such evidence).

Hopefully I've explained myself adequately now.

Comment author: Tyrrell_McAllister 31 January 2011 06:00:30AM * 1 point [-]

E is "we have seen no subversive activity".

I don't like the way you describe that. It is confusing. The evidence is subversive activity. You cannot go out and look for no subversive activity, that makes no sense.

Our evidence is always only what we have observed. Maybe it is strange to say that you "looked for no subversive activity". But you certainly can look for subversive activity and fail to find it. Not seeing subversive activity when you looked for it is Bayesian evidence. But it would be an error to condition on there being no subversive activity at all, even hidden activity. That would be going beyond your observations. You can only condition on what you saw or didn't see when you looked.

Do you agree that, under some priors, you could have p(Q|E) > p(Q) and p(R|E) > p(R), even though Q implies not-R?

Absolutely. I said so two posts up. The question is not about Q and R, though, it's about Q and ~Q.

Okay, I think that we're homing in on the nub of the disagreement.

The propositions in question are

"There is a fifth column that will coordinate a Pearl-Harbor type attack", and
"There is no fifth column".

These are Q and R, respectively. They are not negations of each other. Do you agree?

Comment author: bigjeff5 31 January 2011 05:04:55PM 0 points [-]

Again, I cannot see how you can observe nothing and call it evidence. It is semantics, really, since it makes no difference for the equations, but it makes ~E a positive observation of something and E a negative observation, which is, to me, silly.

These are Q and R, respectively. They are not negations of each other. Do you agree?

Yes. Though, again, I'd rather R be "There is a 5th column" to keep it from being confusing.

With Q there was no evidence that the fifth column was coordinating a timed attack, yet Warren's strongest evidence for it was that there was no evidence for it.

Pearl harbor types of evidence are black swans. You can't just pull them out of the air and add them to your reasoning when you have no solid justification for it. There are a billion other black swans he could have used - what if the Japanese are actually all vampires and had designs on draining the Americans dry? You've got no evidence they aren't, so clearly they are just biding their time!

The former is slightly more reasonable, since something similar had happened recently (though in an entirely different context), but it is no more justified as evidence than the evidence in the vampire scenario.

You must look for other evidence that suggests the 5th column was planning an attack, the fact that you have not been attacked yet is not in any way evidence that they are planning an attack. It is only really evidence that, if they were planning something, they hadn't done it yet. That's all you can get from that - just a guess.

To that end, Warren had no evidence that an American chapter of the 5th Column even existed. There was secret evidence to that effect, but he was not privy to it. He was making the whole thing up because he was afraid.

It was completely unjustified.

Besides, it doesn't make sense. Timed attacks are designed to catch you off guard. After Pearl Harbor, people were always on guard. It wouldn't have had the same effect; a much more effective strategy would have been smaller, guerrilla-type sabotages from within, which they also had zero evidence of.

Comment author: Tyrrell_McAllister 31 January 2011 05:54:25PM * 0 points [-]

Again, I cannot see how you can observe nothing and call it evidence. It is semantics, really, since it makes no difference for the equations, but it makes ~E a positive observation of something and E a negative observation, which is, to me, silly.

Think of ~E as meaning "We observed something, but that 'something' was something other than subversive activity. That is, what we observed was a member of the class of all things that aren't subversive activity."

Is this still "silly"?

Yes. Though, again, I'd rather R be "There is a 5th column" to keep it from being confusing.

This wouldn't change what Warren is saying. It would only change the symbols that we use to restate what he is saying. We would now write ~R to mean "There is no fifth column". So Warren's claim, on my reading, would be

p(Q|E) > P(Q) and p(~R|E) > p(~R).

That is, I would just replace "R" everywhere with "~R". Why is this less confusing? Not observing subversive activity is evidence for there being no fifth column. But it is also evidence for there being a fifth column that is marshaling its resources for a Pearl-Harbor type attack.

Maybe all of these double-negatives are confusing, but that is what the propositional calculus is for: it makes it easy to juggle the negatives just like negation signs in algebra.

Comment author: bigjeff5 01 February 2011 12:35:22AM * 1 point [-]

My biggest problem with calling a lack of evidence evidence is that it is unnecessary in the first place, which makes it confusing when it comes to discussing it.

Also, I'm not arguing for or against the existence of the fifth column. I think I was unclear about that earlier, and I think we probably got a signal or two crossed. The fifth column was a fact, it existed in Japan, and it is the reason they were afraid of a fifth column in America.

Warren also never argued their existence, only their activity, so I don't see why you have a Q and an R at all. Re-read the statement, he took the 5th column's existence as a given.

What I'm arguing is the idea that a lack of evidence of subversive activity can be strong evidence that a plan similar to Pearl Harbor is being hatched.

To that end, I went ahead and made some calculations.

These are my assumptions, and I feel they are historically reasonable (I didn't cite studies, so I can't exactly call them accurate):

1% of all subversive plots are surprise plots (a-la Pearl Harbor). I call these p(subversion).

Evidence for such plots I call p(evidence).

90% of the time when there is such a plot, there is evidence of it before the fact. I call this p(evidence|subversion).

This is the critical part of Warren's statement - he is essentially assuming the opposite of what I say here, and I assert this is not reasonable given what we know of such plots. There was even evidence of the Pearl Harbor plot before hand. An attack was expected and planned for; it was really only the location (and the lack of a prior declaration) and precise timing that was a surprise militarily. I've frankly never heard of a case of a surprise attack with absolutely no evidence that it would occur, so I believe I am being extremely generous with this number. I would not accept lowering this number much further than this.

Last, I assert that 5% of the time when evidence is found for subversion, no subversion actually occurs. Again, I think this is a reasonable number, and probably too low. I wouldn't have a problem adjusting this number down as low as 1%.

Everything else is calculated based on these three assumptions.

p(subversion) = 1% p(~subversion) = 99%

p(evidence|subversion) = 90% p(~evidence|subversion) = 10% p(evidence|~subversion) = 4.95% p(~evidence|~subversion) = 94.05%

p(evidence) = 5.85% p(~evidence) = 94.15%

p(subversion|evidence) = 15.52% p(~subversion|evidence) = 35.35% p(subversion|~evidence) = 0.11% p(~subversion|~evidence) = 99.89%

So my conclusion on the question of how likely a lack of evidence implies a plot for subversion is drawn from the last two figures. Given my assumptions, which I believe are consistent with history, 99.89% of the time when there is no evidence of a plot for a surprise attack, there is no plot for a suprise attack. This means 0.11% of the time when there is no evidence of a plot, there actually is a plot.

Thus, likelihood of a Pearl Harbor style plot when there is no evidence to that fact is 0.11%.

continue this thread »

Comment author: Strange7 19 August 2011 01:01:42AM 1 point [-]

Again, I cannot see how you can observe nothing and call it evidence.

He didn't observe "nothing." He observed factories and shipyards and so forth, continuing to operate without apparent sabotage.

bigjeff5 comments on Absence of Evidence Is Evidence of Absence - Less Wrong

Article Navigation

Comments (108)

Nearest Meetups

Virtual Study Room

Recent Comments

Recent Posts

Recent Rationality Quotes

Recent Wiki Edits

Top Contributors, 30 Days

Recent Karma Awards

Comments (108)

Top Contributors, 30 Days

You'll need to login or register to do that

(Don't worry, it only takes a few seconds)

Create

Login