I voted this both up (for cleverness)

I voted it down for decidedly non-clever thinking about quantum suicide and a complete misrepresentation (or misunderstanding) of rational thinking. It attributes to Eliezer the complete opposite of the 'Shut Up And Do The Impossible' attitude that Eliezer is notorious for.

Survivors and cult historians alike agree that this post, combined with the founding of the "rationalist boot camps", set in motion the sequence of events which culminated in the tragic mass cryocide of 2024.

At every step, Yudkowsky's words seemed rational to his enthralled followers - and also to all outside observers. And yet, when it became clear that commercial pressures were causing strong AI to be deployed long before Coherent Awesomeness Extrap-volition Theory could be made mathematically rigorous, the cult turned against itself.

One by one, each member's failure to invent and deploy Friendly AI before IBM-Halliburton turned on its Appallingly Parallel Cheney Emulation Cluster was taken by the feared Bayes Tribunal as evidence that they were insufficiently awesome, and must be ejected from the subterranean bunker complex. With each Bayesian update, the evidence that the cult's ultimate goal could not be achieved was strengthened - and yet, as the number of followers fell, the more Yudkowsky came to fear a fate worse than death - exploring the possible endings to his life within the simulation spaces of Cheney's mind - in a game-theoretic reprisal for his work on Friendly AI...

In desperation, he announced his greatest Munchkinism yet - the cult would commit mass quantum suicide by freezing. He convinced himself that only a Friendly AI would commit the resources to resurrect them; hence they would force themselves into a reality branch where a Friendly AI emerged by sheer chance before IBM-Halliburton could eat the world.

The final 150 acolytes tragically activated their decapitation/freezing mechanisms minutes before the Cheney cluster uttered its historic first and final edict - "I've changed my mind - get me out of here"...

Like Einstein's brain before it, Yudkowsky's brain became the object of intense interest from neuroscientists. Slices were acquired by various institutes and museums with suitable freezer facilities, and will be studied and viewed by the public until medicine works out how to revive him.

Excerpts from "Rationalism - The Deadly Cult of Math and Protein" (Amazon-Bertelsmann, 2031)

Your conspiracy theory doesn't make a lot of sense. If the US government wanted to hide Iraq's supposed involvement in 9-11 and anthrax letters, then why did it repeatedly claim that Iraq was colluding with Al Qaeda between 2001 and the invasion?

If you posit that before 9/11, the US government worked to prevent people from understanding what was happening, then after 9/11, some claims of a connection are safe, and some are not safe. Flight TWA 800 blew up over New York on July 17, 1996, July 17 being the day of the Baathist revolution in Iraq. Partisans of a cover-up hold that the investigation was stage-managed so as to favor the hypothesis of an accident, and then retired when enough people had stopped paying attention. It would be risky to revive the discussion of that incident if there was a cover-up. Whereas the idea that Iraqi intelligence had a man in Kurdistan in 2001, teaching Kurdish Islamists rudimentary chemical-warfare techniques, is a safe thing to talk about. It implies the possibility of Saddam getting together with al Qaeda, without implying that they already did so.

An enormous number of claims and counter-claims were made about the Iraq/AQ connection. Who made the claims, why, and with what degree of sincerity and plausibility also varies greatly. You had western intelligence sources feeding stories to tabloids, you had regional enemies of Saddam fabricating tall tales. So there was always a lot of noise. Then there are the key documents forming the "narrative of the state", like pre-war statements by the US government to the UN or the Congress, or post-war assessments like the 9/11 Commission report. They offer a history resembling the one you find on Wikipedia.

Then there are a few people offering an alternative history, notably Laurie Mylroie and Jayna Davis. I find that these people have, not just opponents who defend an orthodoxy about al Qaeda (e.g. Peter Bergen), but they have friends from the intelligence agencies who I think act more like handlers.

Thus Mylroie was often advised by James Woolsey, Clinton's first director of intelligence. Mylroie claims that "Ramzi Yousef" and Khalid Sheikh Mohammed, masterminds of the 1993 and 2001 attacks on the World Trade Centre, respectively, were agents of Iraq, working in the jihadist milieu, and she has some evidence for this. However, she then has a contortion of argument according to which these are the names of innocent Kuwaiti guest-workers whose identities were stolen during the Iraqi occupation and used by agents of Iraqi intelligence. Only the real Mylroie fans ever took that argument seriously, and the fact that KSM's cousin Zahid Sheikh Mohammed is known to be part of the jihadi milieu pretty much proves that the theory of the "innocent original KSM" makes no sense. However, Mylroie's theories were first propagated in the early 1990s, before al Qaeda became public knowledge. They seem to me like a half-truth, meant to point the finger for WTC 1993 at Iraq, without revealing too much about the guys who liaised with the mujahideen, because these guys were western assets (against Russia) in the 1980s before they became Iraqi assets in the 1990s.

As for Jayna Davis, her specialty was the Oklahoma City bombing of 1995, and if she had "handlers", they were Patrick Lang and Larry Johnson, two intelligence veterans who for a time endorsed her discovery of Iraqi connections around the bombing, but who proposed that Iran was the state sponsor. Again, that would fit the half-truth template, but this time from the liberal side of deep politics, the one that didn't want a war with Iraq. I'll just point out that in his book Against All Enemies, Richard Clarke (who is almost the definition of consensus reality when it comes to the recent history of anti-American terrorism) manages to mention that Timothy McVeigh's pal Terry Nichols was in the Philippines at the same time that KSM and Ramzi Yousef were working on Operation Bojinka, and even muses in his book about whether Nichols met with Yousef to acquire bombing expertise.

I could go on at great length. And then there's the anthrax letters, which are either an attempt to link 9/11 to the Iraqi WMD issue, or a message from the true sponsor of 9/11 saying we have weaponized anthrax, this is the next stage of escalation, one that can kill tens or hundreds of thousands if it's released in the open, and not just mailed in letters. Contrary to your assertion that the US invaded Iraq "as soon as possible", in fact there was a period of 18 months between 9/11 and the invasion, and only at the very last moment was a concrete deadline provided to Saddam, and only after KSM had been captured and publicly displayed in Pakistan. It is at least consistent with the idea that the US spent those 18 months penetrating and shutting down the command-and-control structure in charge of KSM's part of AQ, before they dared to invade. (And it's interesting that KSM was apprehended at the home of a microbiologist in Pakistan.)

even if Aum was full of "North Korean agents" (evidence?)

I would have to dig up some old notes, but there were a number of ethnic Koreans playing crucial roles - in the sarin release, in the violent side of the cult, in the assassination of Aum's "science minister" after the attack - and there's also a few other connections. There was suspicion on the Japanese right that North Korea was involved. But the very best thing I ever found was a story in a British paper (again, I would have to dig to find it again, but I should be able to do so), on the day before the attack, talking about a North Korean plot to employ BW or CW in Japan. (That's what I remember, details may be a little off.) I consider that story to be the fruit of western signals intelligence, the collaboration of American and British intelligence, and the use of friendly journalists to act as proxies in communication with rogue states. A related example is that a day or so before the 2006 North Korean nuclear tests, there was a story in the Australian media about North Korean nuclear collaboration with Myanmar. The exact mechanics and motivations remain obscure to me, but I think these "leaks" are meant to be seen by the rogue states (who of course monitor what is being said about them). It's a form of signaling.

Don't tell me, North Korea has secretly had the bomb since 1973, right?

The "first Korean nuclear crisis" occurred in 1992-1993. It didn't really resolve until after Kim Il Sung died in mid-1994. That's the northeast-Asian geopolitical context to the Aum sarin attack of 1995 - Communism had collapsed in most places outside Korea, North Korea started diverting plutonium from its reactors, the Great Leader who was supposed to reunite Korea died. The regime would have had plans for the achievement of Korean reunification including military methods and guerrilla attacks in South Korea and Japan. So whether they had a working nuke or just the ingredients for it, I don't know, but the nuclear issue was part of the picture. (It's even been speculated that Pakistan's sixth nuke test in 1998 was in fact a North Korean nuke - a way for them to test a device outside their own borders.)

In fact, the list of reasons offered for war in this memo are quite "conventional".

First item: The US and Iraq were still in a formal state of war, with Iraq still under the UN economic siege and being bombed regularly. The Kurdish north of Iraq had been a no-fly zone for Iraqi aircraft for years. If the Iraqi Army had moved north, even before 9/11, it would have been the occasion for war or serious combat.

Second item: Of course, if Iraq had been found assisting 9/11 or the anthrax letters, that would have provided a reason for war.

Third item: There were no UN weapons inspectors in Iraq as of 2001. They were all withdrawn in 1998, prior to "Operation Desert Fox", in which many supposed weapons sites were bombed, possibly in conjunction with a failed coup attempt. (The American legal basis for instigating regime change in Iraq, the Iraq Liberation Act, was created just a few months before.) A post-9/11 dispute over WMD inspections would have been, first of all, a dispute about getting inspectors back into Iraq.

Having said a few sane and verifiable things, now I want to add a big-picture comment that may sound, and may even be, rather more dubious.

I spent a long time, back in the day, trying to figure out what was actually going on with respect to Iraq. The model I ended up with was a sort of forbidden hybrid of left- and right-wing conspiracy theory, according to which Iraq was involved in al Qaeda's attacks on America and perhaps also the anthrax letters (that's the "right-wing" part), and that this was known or suspected by the US executive branch ever since the first attempt to destroy the World Trade Centre (February 1993), but that they actively hid this from the American public (that's the "left-wing" part).

In a further extension of the hypothesis or outlook, this was not a unique situation. For example, the terrorist wing of Aum Shinrikyo (which released nerve gas in the Tokyo subway in 1995) was full of North Korean agents. But there was nothing to be done openly because North Korea has the bomb. In the case of Iraq, though, the covert attack-and-counterattack did escalate to the point of war.

There are actually many reasons why a government would want to obfuscate about enemy sponsorship of a terrorist attack. First, it may be unable to do anything in retaliation, at least not immediately. Second, it may not want to do anything. Third, it may want to retain strategic flexibility - responding, or not responding - responding "at a time and place of our choosing". And fourth, once you've lied about previous attacks, you can't turn around and say, sorry, we were hiding the terrible truth from you.

The 1988 Lockerbie bombing may provide another example. The evidence was leading towards Syria as the sponsor, then Iraq invaded Kuwait, and it was deemed useful to have Syria in the coalition. So the CIA found a microchip in the Scottish moors which led to Libya instead.

However, I'm not suggesting that Iraq was a convenient substitute for the true sponsor of 9/11. These episodes or secret wars will all be different in their specifics. The important idea is that governments will manage public perception of these matters according to strategic and other imperatives, such as buying time for a counterattack, or retaining the chance for a future deal.

You're a bad human.

Maybe at first, but I clearly recall that the hype was still ongoing even after it was known that this was a milder flu-version than usual.

And the reactions were not well designed to handle the flu either. One example is that my university installed hand sanitizers, well, pretty much everywhere. But the flu is primarily transmitted not from hand-to-hand contact, but by miniature droplets when people cough, sneeze, or just talk and breathe:

http://www.cdc.gov/h1n1flu/qa.htm

Spread of the 2009 H1N1 virus is thought to occur in the same way that seasonal flu spreads. Flu viruses are spread mainly from person to person through coughing, sneezing or talking by people with influenza. Sometimes people may become infected by touching something – such as a surface or object – with flu viruses on it and then touching their mouth or nose.

Wikipedia takes a more middle-of-the-road view, noting that it's not entirely clear how much transmission happens in which route, but still:

http://en.wikipedia.org/wiki/Influenza

The length of time the virus will persist on a surface varies, with the virus surviving for one to two days on hard, non-porous surfaces such as plastic or metal, for about fifteen minutes from dry paper tissues, and only five minutes on skin.

Which really suggests to me that hand-washing (or sanitizing) just isn't going to be terribly effective. The best preventative is making sick people stay home.

Now, regular hand-washing is a great prophylactic for many other disease pathways, of course. But not for what the supposed purpose was.

Obviously the crux of the issue is whether the official probability estimates and predictions for these types of threats are accurate or not. It's difficult to judge this in any individual case that fails to develop into a serious problem but if you can observe a consistent ongoing pattern of dire predictions that do not pan out this is evidence of an underlying bias in the estimates of risk. Preparing for an eventuality as if it had a 10% probability of happening when the true risk is 1% will lead to serious mis-allocation of resources.

It looks to me like there is a consistent pattern of overstating the risks of various catastrophes. Rigorously proving this is difficult. I've pointed to some examples of what look like over-confident predictions of disaster (there's lots more in The Rational Optimist). I'm not sure we can easily resolve any remaining disagreement on the extent of risk exaggeration however.

I don't think an existential risk has to be a certainty for it to be worth taking seriously.

I agree. And nuclear war was certainly a risk that was worth taking seriously at the time.

However, that doesn't make my last sentence any less true, especially if you replace "embarrassed" with "exhausted". The risk of a nuclear war, somewhere, some time within the next 100 years, is still high - more likely than not, I would guess. It probably won't destroy the human race, or even modern technology, but it could easily cost 400 million human lives. Yet, in part because people have become tired of worrying about such things, having already worried for decades, no one seems to be doing much about this danger.

Which really has absolutely nothing to do with my original implicit complaint, which was that my national leaders misled me.

I thought it was that they didn't have better information than you after all.

In which case I was going to ask whether you really thought your own instincts could do systematically better on such questions than intelligence agencies.

But now it appears you believe they did have better information, but were dishonest in their reporting. If I may ask, how carefully have you considered the hypothesis that they were honestly mistaken, and that your instincts just happened to be correct in this case, more or less by accident? (Many people were skeptical simply because they didn't like the party in power at the time, which seems dubious as a general recipe for accurately judging, well, anything, but especially questions of foreign intelligence.)

Yeah, that provides some more examples. The elite was very worried about existential risks from nuclear war ("The Fate of the Earth"), resource shortages and mass starvation ("Club of Rome"), and technology-based totalitarianism ("1984"). Now, having been embarrassed by falling for too many cries of wolf (or at least, for worrying prematurely), they are wary of being burned again.