Recent brainstorming sessions at SIAI (with participants including Anna, Carl, Jasen, Divia, Will, Amy Willey, and Andrew Critch) have started to produce lists of rationality skills that we could potentially try to teach (at Rationality Boot Camp, at Less Wrong meetups, or similar venues). We've also been trying to break those skills down to the 5-second level (step 2) and come up with ideas for exercises that might teach them (step 3) although we haven't actually composed those exercises yet (step 4, where the actual work takes place).
The bulk of this post will mainly go into the comments, which I'll try to keep to the following format: A top-level comment is a major or minor skill to teach; upvote this comment if you think this skill should get priority in teaching. Sub-level comments describe 5-second subskills that go into this skill, and then third-level comments are ideas for exercises which could potentially train that 5-second skill. If anyone actually went to the work of composing a specific exercise people could run through, that would go to the fourth-level of commenting, I guess. For some major practicable arts with a known standard learning format like "Improv" or "Acting", I'll put the exercise at the top and guesses at which skills it might teach below. (And any plain old replies can go at any level.)
I probably won't be able to get to all of what we brainstormed today, so here's a PNG of the Freemind map that I generated during our session.
Subskill: Avoid pitfalls of verbal strategic reasoning.
There are studies showing that people who consider their decisions more make worse decisions. As I understand it, the main explanation for this is that people shift from an intuitive sum of costs and benefits, to seeking verbally justifiable decisions, which in turn might lead them to one-reason-decisionmaking, ignoring some of their costs and benefits which are important to them but seem less "sensible". I also suspect it may exacerbate other biases like scope insensitivity or rare events - thinking about cases which are rare or short in duration.
The classic case being "Let's get a bigger house, further away from work, so it has an extra bedroom in case Grandma comes over", which she does once a year, but the 20 minutes of extra commute time happen every day and are not acclimated-to.
"Let's get a bigger house, further away from work, so it has an extra bedroom in case Grandma comes over"
Not saying this is a bad example, but it COULD be the case that grandma never being able to come over is totally unacceptable. Which is also a pitfall - something can seem trivial until it goes away.