Tetlock wrote in Superforcasters that the US intelligence establishment was likely justified to believe that it was likely that Iraq was hiding WMDs. According to Tetlock their sin was that they asserted that it's certain that Iraq had WMD.

When first reading Superforcasters I didn't quite understand the situation. After reading https://theintercept.com/2015/04/10/twelve-years-later-u-s-media-still-cant-get-iraqi-wmd-story-right/ I did.

The core problem was that Saddam lost track of some of his chemical weapons. His military didn't do perfect accounting of them and they looked the same as conventional weapons. It takes an x-ray to tell his chemical weapons apart from the normal ones.

The US intercepted communications where Saddam told his units to ensure that they had no chemical weapons that inspectors could find. Of course, that communication didn't happen in English. That communication seems to have been misinterpreted by the US intelligence community as evidence that Saddam is hiding WMDs.

Nearly nobody understood that Iraq having chemical weapons and hiding them are two different systems because you need to know where your chemical weapons happen to be to hide them. On the same token, nobody publically argues that pure incompetence might be the cause of chemical weapon usage in Syria. We want to see human agency and if a chemical weapon exploded we want to know that someone is guilty of having made the decision to use them.
In a recent facebook discussion about Iraq and the value of IARPA, a person asserted that the US intelligence community only thought Iraq had WMDs because they were subject to political pressure. 
We have to get better at understanding that bad events can happen without people intending them to happen.

After understanding Iraq it's interesting to look at Syria. Maybe the chemical weapons that exploded in Syria didn't explode because Assad's troops or the opposition wanted to use chemical weapons. They might have simply exploded because some idiot did bad accounting and mislabeled a chemical weapon as being a conventional weapon.

The idea that WMDs explode by accident might be too horrible to contemplate. We have to be better at seeing incompetence as a possible explanation when we want to pin the guilt for having made an evil decision on another person.

New Comment
10 comments, sorted by Click to highlight new comments since: Today at 10:18 AM

Or some underpaid and enterprising manager of a conventional weapons depot in Iraq sold Assad resupplies, or ISIS captured depots ended up on the black market and bought by Assad's middlemen. And then what happened in Syria was not only a tragic accident but those missing Iraqi WMDs.

The question "Did Iraq have chemical weapons?" is a stupid question because it is not quantified. The inspectors gave a small upper bound on the number of weapons and they were right.

Yes, the CIA reached its conclusions from its communications interceptions. But just because they had reasons for those conclusions doesn't mean that they were good reasons. It should instead have trusted the inspectors, who actually knew something about weapons, unlike the CIA. Chemistry trumps spies.

There is a lot of equivocation between weapons and factories. People summarize the question as "Did Iraq have WMD?" but what Powell said to the UN was that Iraq had chemical and biological weapons factories. This was completely wrong. If the CIA can't remember what it said, it can't learn from its mistakes. But if its job is to fabricate causus belli, heck of job.

My father had a hypothesis about WMDs in Iraq: Saddam's underlings were corrupt and "exaggerated" the amount of chemical weapons they had produced. After all, anyone who admitted that he couldn't make the quantity he promised was going to be in big trouble. Also, if you say you made 100 units of chemical weapons but only made 50, you get paid for making 100 but only had to spend enough money to make 50, and you get to steal the difference. Again, nobody is going to admit to stealing from Saddam, so when it came time to destroy the 100 units that existed on paper, the inspectors would only find the 50 that actually existed, and there would be 50 more units "missing" that nobody could admit didn't exist in the first place.

For some weapons that might indeed have been the case but when the weapons looking the same as conventional weapons it's not far fetched that some really went missing.

After all even the big powers misplaced up to 50 nuclear warheads during the cold war.

The US intercepted communications where Saddam told his units to ensure that they had no chemical weapons that inspectors could find. Of course, that communication didn't happen in English. That communication seems to have been misinterpreted by the US intelligence community as evidence that Saddam is hiding WMDs.

Even in the English given, I can see alternate interpretations. Make sure you destroy any you have. Make sure they can't find any you have.

Anyone knows how intelligence works? Given those two interpretations, do they get assigned priors that trickle up?

My impression from the Saddam days is that the scenario that Saddam destroyed his weapons the best he could, while trying to maintain the appearance that he had them, really hadn't trickled up to general consciousness, if anyone had considered it at all.

People are overly confident in thinking they've covered the possible motivations a person might have had. "He wouldn't have done this unless."

Any actual science out there on this particular effect?

Yeah, the story I heard was that he probably wanted Iran to think he still had them...

Anyone knows how intelligence works? Given those two interpretations, do they get assigned priors that trickle up?

If we can believe Tetlock, the US intelligence community didn't assign any probabilities to the interpretations. Afterwards they felt like the screwed up and funded projects like the ACE challenge that Tetlock's Good Judgement Project won via IARPA to find out how things should be done.

The problem with probabilities is that that they produce accountability. If you say X is likely to happen and X happens you can pretend that you meant that it happened with 95% probability and if X doesn't happen you can pretend that your prediction was that it was supposed to happen with 60% probability. As a result there are institutional incentives against using probability.

I don't know what the status quo happens to be within the intelligence community but when I look at the current US administration, I doubt that pushing the CIA to use probabilities more for their assessments is high on the list of priorities.

At the same time getting intelligence right in cases like this is important. That's part of the reason why IARPA spending isn't as evil as weapons research and why 80,000 hours is justified in putting IARPA-Program Manager on their job board.

Did they even have "Saddam is faking it" as a possibility?

Saddam had WMD's, after all. Kim has it, too. I wouldn't be at all surprised, if it is quite common. A little secret of a lot of countries, worldwide. We shall see what some neural network, or some other system will have to say about this matter.

The differs is that Saddam didn't want to have WMD's at the time and wasn't hiding WMD's while Kim is happy with the WMD's he has and is open about being proud of having them.