Logos, you don't need to preach about utilitarian calculations to us. You have it the other way around. We don't condemn your words because we can't make them, we condemn them because we can make them better than you. ( Note particularly in this case the willful refusal to accept the counterfactual and the accusations of irresponsibility for "not emphasizing strongly enough" skepticism in reaching conclusions. )
No, what's going on here is something significantly "other" than "everyone here knows it wasn't actually a death threat [...] but wanted to warn you it could be perceived that way." -- those are mutually exclusive conditions by the way; either everyone does not know this, or it can't be perceived that way.
The truly ironic thing is that there isn't a legitimate interpretation of my words that could make them a death threat. I responded to an initial counterfactual with a query as to the moral justification of refusing to take individual action in an end-of-the-world-if-you-don't scenario.
In attempting to explore this, I was met with repeated willful refusals to engage the scenario, admonitions to "not be creepy", and bald assertions that "I'm not better at moral calculus but worse".
These responses, I cannot help but conclude, are demonstrative of cached moral beliefs inducing emotional responses overriding clear-headed reasoning. I'm used to this; the overwhelming majority of people are frankly unable to start from the 'sociopathic' (morally agnostic, that is) view and work their way back to a sound moral epistemology. It is no surprise to me that the population of LW is mainly comprised of "neurotypical" individuals. (Please note: this is not an assumption of superiority on my part.)
This is unfortunate, but... short of 'taking the karma beating' there's really no way for me to demonstratively point that out in any effective way.
I don't think I'm going to continue to respond any further in this thread, though. It's ceased being useful to any extent, insofar as I can see.
Here's a poser that occurred to us over the summer, and one that we couldn't really come up with any satisfactory solution to. The people who work at the Singularity Institute have a high estimate of the probability that an Unfriendly AI will destroy the world. People who work for http://nuclearrisk.org/ have a very high estimate of the probability that a nuclear war will destroy the world (by their estimates, if you are American and under 40, then nuclear war is the single most likely way in which you might die next year).
It seems like there are good reasons to take these numbers seriously, because Eliezer is probably the world expert on AI risk, and Hellman is probably the world expert on nuclear risk. However, there's a problem - Eliezer is an expert on AI risk because he believes that AI risk is a bigger risk than nuclear war. Similarly, Hellman chose to study nuclear risks and not AI risk I because he had a higher than average estimate of the threat of nuclear war.
It seems like it might be a good idea to know what the probability of each of these risks is. Is there a sensible way for these people to correct for the fact that the people studying these risks are those that have high estimate of them in the first place?