Carl, following that link to its source brought me here: http://www.nice.org.uk/guidance/index.jsp?action=download&o=33188, where several randomized trials are mentioned. But I see no meta-analysis, so I still worry about publication selection biases, etc. Anyone know of a meta-analysis of this lit?
Rational Me or We?
Martial arts can be a good training to ensure your personal security, if you assume the worst about your tools and environment. If you expect to find yourself unarmed in a dark alley, or fighting hand to hand in a war, it makes sense. But most people do a lot better at ensuring their personal security by coordinating to live in peaceful societies and neighborhoods; they pay someone else to learn martial arts. Similarly, while "survivalists" plan and train to stay warm, dry, and fed given worst case assumptions about the world around them, most people achieve these goals by participating in a modern economy.
The martial arts metaphor for rationality training seems popular at this website, and most discussions here about how to believe the truth seem to assume an environmental worst case: how to figure out everything for yourself given fixed info and assuming the worst about other folks. In this context, a good rationality test is a publicly-visible personal test, applied to your personal beliefs when you are isolated from others' assistance and info.
I'm much more interested in how we can can join together to believe truth, and it actually seems easier to design institutions which achieve this end than to design institutions to test individual isolated general tendencies to discern truth. For example, with subsidized prediction markets, we can each specialize on the topics where we contribute best, relying on market consensus on all other topics. We don't each need to train to identify and fix each possible kind of bias; each bias can instead have specialists who look for where that bias appears and then correct it.
Perhaps martial-art-style rationality makes sense for isolated survivalist Einsteins forced by humanity's vast stunning cluelessness to single-handedly block the coming robot rampage. But for those of us who respect the opinions of enough others to want to work with them to find truth, it makes more sense to design and field institutions which give each person better incentives to update a common consensus.
In response to Dialectical Bootstrapping
I suspect you are right; the issue isn't that these people haven't "learned" relevant abstractions or tools. They just don't have enough incentives to apply those tools in these context. I'm not sure you "teach" incentives, so I'm not sure there is anything you can teach which will achieve the goal stated. So I'd ask the question: how can we give people incentives to apply their tools to cases like religion?
In response to Beginning at the Beginning
In my recent blogging heads TV with Tyler Cowen we argued about the value of "having explicit and conscious standards of validity, and applying them in a systematic way." My intuition agrees with you, but can we point to anything more concrete than that? Do we have data suggesting this approach is in fact more accurate?
In response to Don't Believe You'll Self-Deceive
It seems to me you are trying to deceive yourself into thinking that you cannot comfortably self-deceive. Your effort may indeed make it harder to self-deceive, but I doubt it changes your situation all that much. Admit it, you are human, and within the usual human range of capabilities and tendencies for self-deception.
In response to Kinnaird's truels
"That's what happens to a field when it unbinds itself from the experimental evidence" - so the million dollar question for Less Wrong is: what experimental evidence can this community bind itself to, to avoid the same outcome?