On at least two occasions - one only a year past - my life was at serious risk because I was not thinking clearly.  Both times, I was lucky (and once, the car even survived!).  As a gambler I don't like counting on luck, and I'd much rather be rational enough to avoid serious mistakes.  So when I checked the top-ranked posts here and saw Robin's Rational Me or We? arguing against rationality as a martial art I was dumbfounded.  To me, individual rationality is a matter of life and death[1].

In poker, much attention is given to the sexy art of reading your opponent, but the true veteran knows that far more important is the art of reading and controlling yourself.  It is very rare that a situation comes up where a "tell" matters, and each of my opponents is only in an occasional hand.  I and my irrationalities, however, are in every decision in every hand.  This is why self-knowledge and self-discipline are first-order concerns in poker, while opponent reading is second or perhaps even third.

And this is why Robin's post is so wrong[2].  Our minds and their irrationalities are part of every second of our lives, every moment we experience, and every decision that we make.  And contra to Robin's security metaphor, few of our decisions can be outsourced.  My two bad decisions regarding motor vehicles, for example, could not have easily been outsourced to a group rationality mechanism[3].  Only a tiny percentage of the choices I make every day can be punted to experts.

Martial arts can be a good training to ensure your personal security, if you assume the worst about your tools and environment.  If you expect to find yourself unarmed in a dark alley, or fighting hand to hand in a war, it makes sense.  But most people do a lot better at ensuring their personal security by coordinating to live in peaceful societies and neighborhoods; they pay someone else to learn martial arts.  Similarly, while "survivalists" plan and train to stay warm, dry, and fed given worst case assumptions about the world around them, most people achieve these goals by participating in a modern economy.

The martial arts metaphor for rationality training seems popular at this website, and most discussions here about how to believe the truth seem to assume an environmental worst case: how to figure out everything for yourself given fixed info and assuming the worst about other folks.  In this context, a good rationality test is a publicly-visible personal test, applied to your personal beliefs when you are isolated from others' assistance and info.  

I'm much more interested in how we can can join together to believe truth, and it actually seems easier to design institutions which achieve this end than to design institutions to test individual isolated general tendencies to discern truth.  For example, with subsidized prediction markets, we can each specialize on the topics where we contribute best, relying on market consensus on all other topics.  We don't each need to train to identify and fix each possible kind of bias; each bias can instead have specialists who look for where that bias appears and then correct it. 

Perhaps martial-art-style rationality makes sense for isolated survivalist Einsteins forced by humanity's vast stunning cluelessness to single-handedly block the coming robot rampage.  But for those of us who respect the opinions of enough others to want to work with them to find truth, it makes more sense to design and field institutions which give each person better incentives to update a common consensus.