Recent brainstorming sessions at SIAI (with participants including Anna, Carl, Jasen, Divia, Will, Amy Willey, and Andrew Critch) have started to produce lists of rationality skills that we could potentially try to teach (at Rationality Boot Camp, at Less Wrong meetups, or similar venues). We've also been trying to break those skills down to the 5-second level (step 2) and come up with ideas for exercises that might teach them (step 3) although we haven't actually composed those exercises yet (step 4, where the actual work takes place).
The bulk of this post will mainly go into the comments, which I'll try to keep to the following format: A top-level comment is a major or minor skill to teach; upvote this comment if you think this skill should get priority in teaching. Sub-level comments describe 5-second subskills that go into this skill, and then third-level comments are ideas for exercises which could potentially train that 5-second skill. If anyone actually went to the work of composing a specific exercise people could run through, that would go to the fourth-level of commenting, I guess. For some major practicable arts with a known standard learning format like "Improv" or "Acting", I'll put the exercise at the top and guesses at which skills it might teach below. (And any plain old replies can go at any level.)
I probably won't be able to get to all of what we brainstormed today, so here's a PNG of the Freemind map that I generated during our session.
Exercise A: For a case where sunk costs exist, imagine that you are a new person who was just now teleported into this person's life. Variant: Imagine you were just teleported into this person's life, and everyone else knows this, and they all expect you to execute sudden changes of course. See what this changes about your thinking, regardless of whether it changes your decision.
Exercise B: For whatever sunk costs you're in the middle of expending, look at the same scenario and rephrase it as a sunk benefit, the purchased option to finish a task more quickly than before. E.g., if you already paid $100 on a $150 item, change from "I paid $100" to "I now have the option of purchasing this item for $50". Again, the stated objective of the exercise will be, "Notice the difference in your thinking", not, "try to change your decision".
Exercise C: As above, but imagine that you bought the option for a penny on eBay. E.g. if you're one year into a four-year PhD program, imagine that you paid a penny on eBay to purchase an option to get a PhD in three years rather than the usual four years. Would you exercise that option if you paid a penny for it?
That last exercise seems to run afoul of some value-related heuristics. Price is so common a proxy for utility that imagining that you paid a penny for the option on eBay might irrationally devalue whatever you're looking at.
Of course, looking at the contrast might still give you some useful insights -- provided you can untangle them.