A few minor clarity/readability points:

1. The second paragraph opening "The statisticians who discovered the nature of reality" reads rather oddly when taken out of the context of "The Fabric of Real Things".
2. When considering the three causal models of Burglars, Alarms and Recessions, tackling the models in a "First, third, second" order threw me on first reading. It would probably be easier to follow if the text and the diagram used the same order.
3. Perhaps giving each node a different pastel colour would make it easier to follow what is changing between different diagrams.

And this has probably been said, but using exercise and weight is probably distracting, since people already have opinions on the issue.

All in all though, a great article.

I know as a matter of process that when a respected fellow rationalist tells me that I need to become curious, I should pause and check my curiosity levels and try to increase them.

How does one increase their curiosity levels?

I use "I think X" to indicate more uncertainty than just "X" all the time, and so does Eliezer. I just checked his recent comments, and on the first page, he used "I think" two times to indicate uncertainty, and one time to indicate that others may not share his belief. The statement "Consider the following deflations, all of which convey essentially the same information about your own opinions" just seems plain wrong.

One of the most audacious and famous experiments is known informally as "the door study": an experimenter asks a passerby for directions, but is interrupted by a pair of construction workers carrying an unhinged door, concealing another person whom replaces the experimenter as the door passes. Incredibly, the person giving directions rarely notices they are now talking to a completely different person. This effect was reproduced by Derren Brown on British TV (here's an amateur re-enactment).

I think the response of the passerby is quite reasonable, actually. Confronted with a choice between (a) "the person asking me directions was just spontaneously replaced by somebody different, also asking me directions," and (b) "I just had a brain fart," I'll consciously go for (b) every time, especially considering that I make similar mistakes all the time (confusing people with each other immediately after having encountered them). I know that this is probably not a phenomenon that occurs at the conscious level, but we should expect the unconscious level to be even more automatic.

The criticism is that a martial artist or scientist is actually trying to attain a highly specific brain-state in which neurons have particular patterns in them; a feeling of emptiness, even if part of this brain state, is itself a neural pattern and certainly does not correspond to the absence of a mind.

The zeroth virtue or void - insofar as we believe in it - corresponds to particular mode of thinking; it's certainly not an absence of mind. Emptiness, no-mind, the Void of Musashi, all these things are modes of thinking, not the absence of any sort of reified spiritual substance. See also the fallacy of the ideal ghost of perfect emptiness in philosophy.

This is actually not that relevant. The only question is whether humanity could survive a Nuclear Winter...

From my own experience of awaking after days of artificial coma I find it unlikely to believe that the person will go through a good Bayesian reasoning.

It's very hard to form the belief that it's Thursday when the last day you remember is a Sunday. It goes against fundamental principles that evolution taught us over millions of years. It's one of those cases where our availability heuristics is really bad.

PS: I know this is only a thought experiment.

I found this post very disturbing, so I thought for a bit about why. It reads very much like some kind of SF dystopia, and indeed if it were necessary to agree to this lottery to be part of the hypothetical rationalist community/country, then I wouldn't wish to be a part of it. One of my core values is liberty - that means the ability of each individual to make his or her own decisions and live his or her life accordingly (so long as it's not impeding anyone else's right to do the same). No government should have the right to compel its citizens to become soldiers, and that's what it would become, after the first generation, unless you're going to choose to exile anyone who reaches adulthood there and then opts out.

Offering financial incentives for becoming a soldier, as has already been discussed in the comments, seems a fairer idea. Consider also that the more objectively evil the Evil Barbarians are, the more people will independently decide that fighting is the better decision. If not enough people support your war, maybe that in itself is a sign that it's not a good idea. If most of the rationalists would rather lose than fight, that tells you something.

It's quite difficult to know the right tone of response to take here - the Evil Barbarians are obviously pure thought-experiment, but presumably most of us would view a rationalist country as a good thing. Not if it made decisions like this, though. Sacrificing the individual for the collective isn't always irrational, but it needs to be the individual who makes that choice based on his or her own values, not due to some perceived social contact. Otherwise you might as well be sacrificed to make more paperclips.

If it was intended as pure metaphor, it's a disquieting one.

Checking up on the guest lists for cocktail parties and customer data for salons, we find that these two activities are indeed disproportionately enjoyed by the rich, so that part of the statement also seems true enough.

P implies Q does not imply Q implies P, surely.

The difference in optimisation targets between LW and H&B researchers is an important thing to point out, and probably the main thing I'll take away from this post.

Biases can:-

• Be interesting to learn about
• Serve an academic/political purpose to research
• Give insight into the workings of human cognition
• Be fun to talk about
• Actually help to achieve your goals by understanding them

And the correlations between any 2 of these things need not be strong or positive.

Is it the halo effect if we assume that a more interesting bias will better help us achieve our goals?