Get into a simulated argument about the issue with someone taking the opposite side from the one you care about.
I think it is important to integrate this with searching for a third alternative.
1) Establish, on paper and before doing anything else, what you think alternative positions are.
2) Have a third party who does not look at your list identify third alternatives.
3) Determine which good ideas you didn't think of. This can only be an approximation, as you and the other person have used different words to describe alternatives. Notice the emotional urge to rationalize and conclude you didn't leave out important third alternatives, and the urge to rationalize that the alternatives you didn't think of aren't compelling. (This is an artificial hot-button issue, but that is only a side-benefit of this step.)
4) The list that you have constructed probably avoids your belief's real weak points. Alternatives identified by the other person, but not you, are best for pitting against your cherished idea, as in exercise C above.
5) Notice the relative strength of the best arguments not on your list as against the ones on your list. If they are strong, consider whether you have been failing to consider third alternatives for intellectual reasons, such as not holding off on proposing solutions, or avoiding your belief's weak points for emotional reasons.
Recent brainstorming sessions at SIAI (with participants including Anna, Carl, Jasen, Divia, Will, Amy Willey, and Andrew Critch) have started to produce lists of rationality skills that we could potentially try to teach (at Rationality Boot Camp, at Less Wrong meetups, or similar venues). We've also been trying to break those skills down to the 5-second level (step 2) and come up with ideas for exercises that might teach them (step 3) although we haven't actually composed those exercises yet (step 4, where the actual work takes place).
The bulk of this post will mainly go into the comments, which I'll try to keep to the following format: A top-level comment is a major or minor skill to teach; upvote this comment if you think this skill should get priority in teaching. Sub-level comments describe 5-second subskills that go into this skill, and then third-level comments are ideas for exercises which could potentially train that 5-second skill. If anyone actually went to the work of composing a specific exercise people could run through, that would go to the fourth-level of commenting, I guess. For some major practicable arts with a known standard learning format like "Improv" or "Acting", I'll put the exercise at the top and guesses at which skills it might teach below. (And any plain old replies can go at any level.)
I probably won't be able to get to all of what we brainstormed today, so here's a PNG of the Freemind map that I generated during our session.