Back when I was (alas) a Christian, I used to say: You should try reading the Bible, at least some of the time, as if you're a skeptic, so that your idea of what it says doesn't get distorted by your need to make it say things you find easy to believe.

(It turns out that that advice is good for other reasons, but that wasn't my point at the time.)

I think this is exactly parallel to Kaj's observation. I think the advice "Learn to love failure" for entrepreneurs is getting at something similar, though my brain's too fuzzy right now to be quite sure.

This technique seems to apply in a whole lot of places. What's its most general statement? Something like this, but I'm not convinced I've got it down to its essence:

When there's some thing X that you trust, so that you aim to make your beliefs match X, beware that doing so may tend instead to distort your estimate of X to match your pre-existing beliefs. One way around this is to allow your beliefs to diverge from X a little, at least temporarily, so that you can figure out what X says without worrying about whether that matches your beliefs.

... At which point it strikes me that allowing your beliefs to diverge from X, when you're sure that X is very reliable, is in fact deliberate irrationality (and, note, not the same deliberate irrationality as Kaj is talking about: they're at different levels of meta-ness, as it were), and that perhaps we have here an example of when deliberate self-deception, or something like it, might help you.