This is a place to talk about hobbies that teach rationality lessons. I'll start with a few:
Programming - Lets you practice math and logic, and gain an intuitive understanding of computation. Also teaches that sometimes you can't argue with reality, but you can fix it. (Also suggested recently by ciphergoth.)
Cryptography - That is, designing/breaking cryptosystems and security protocols. Lets you practice math, logic, and probability theory. Also teaches that almost all human ideas are wrong. After doing this for a while, whenever you encounter a new idea (including your own), you'll instinctively think "If I can't find anything wrong with this, it's probably because I'm not smart or knowledgeable enough or haven't tried hard enough."
Science fiction - Reduces status quo bias and gives interesting insights. Also teaches that the way a society is organized depends a lot on the set of technologies it has access to, so if you don't like how your society works, one lever you have is to change that set.
Video games - Teaches that conventional "success" in life is not much less arbitrary than "winning" in a video game. They're both fine for a diversion, but there are more interesting goals to pursue.
A couple others I've seen suggested in recent comments:
Chess - According to JGWeissman, it "teaches you to carefully consider the consequences of your available actions and choose the action with the best consequences". (I've only played a few games of Chinese Chess, and for me, the lesson was that I don't like competition, and I should look for things to do that nobody else is doing.)
Poker - Lets you practice statistics and overcoming emotional biases.
I tried to justify some of my hobbies to see if I could come up with anything that couldn't be called a "rationalist hobby" to determine if it's a useful designation or not.
Knitting - Trains your attention to fine detail. After you knit a pair of socks, whenever you wear knitted clothes you'll instinctively think, "I could make this. There are no great unknowable secrets in manufacturing, only time and labor."
Music Radio DJing - You learn how to speak fluently and without pause, and put together an entertaining set of music, which are both useful for signalling in social situations.
Reddit - Its up/downvoting system teaches how to quickly decide whether or not something is interesting to you, and the ability to submit content to be judged by the crowds can train your ability to write short copy that will appeal to large audiences.
I feel like I'm stretching, but also like science fiction, video games, chess and poker are probably also stretches.
Building up a really huge record collection - you learn that this is not such a great idea when you have to move house. It's a subclass of this one.