This is a place to talk about hobbies that teach rationality lessons. I'll start with a few:
Programming - Lets you practice math and logic, and gain an intuitive understanding of computation. Also teaches that sometimes you can't argue with reality, but you can fix it. (Also suggested recently by ciphergoth.)
Cryptography - That is, designing/breaking cryptosystems and security protocols. Lets you practice math, logic, and probability theory. Also teaches that almost all human ideas are wrong. After doing this for a while, whenever you encounter a new idea (including your own), you'll instinctively think "If I can't find anything wrong with this, it's probably because I'm not smart or knowledgeable enough or haven't tried hard enough."
Science fiction - Reduces status quo bias and gives interesting insights. Also teaches that the way a society is organized depends a lot on the set of technologies it has access to, so if you don't like how your society works, one lever you have is to change that set.
Video games - Teaches that conventional "success" in life is not much less arbitrary than "winning" in a video game. They're both fine for a diversion, but there are more interesting goals to pursue.
A couple others I've seen suggested in recent comments:
Chess - According to JGWeissman, it "teaches you to carefully consider the consequences of your available actions and choose the action with the best consequences". (I've only played a few games of Chinese Chess, and for me, the lesson was that I don't like competition, and I should look for things to do that nobody else is doing.)
Poker - Lets you practice statistics and overcoming emotional biases.
Improv theater -- Practice in examining and altering your motivations. Sharper-than-usual distinction between terminal goals (humor, drama) and immediate goals (those of your character).
Game design -- Similar benefits to chess. You're in a situation of perfect information and clearly-defined rules, and yet you still bump very quickly against the limits of your own cognition. You're forced to acknowledge and try to work around your limitations, because there are no excuses. Unlike in chess, there are no unsolvable problems; at worst, there are problems with no elegant solution.
Writing pages on the tvtropes wiki -- Heavy overlap with the goals of lesswrong; there are even tvtropes pages for fallacies and biases. Learning to spot patterns in literature has the same challenge as learning to spot them in thought: the most omnipresent ones are also the least visible.
Anything to do with language -- Translation, conlangs, writing in English with arbitrary restrictions...they all help the same way that Rationalist Taboo does.
It also illustrates the gap between what rules directly allow/prohibit, and what they actually incentivize doing. It requires that bit of economic thinking - wha... (read more)