Back when I was (alas) a Christian, I used to say: You should try reading the Bible, at least some of the time, as if you're a skeptic, so that your idea of what it says doesn't get distorted by your need to make it say things you find easy to believe.
(It turns out that that advice is good for other reasons, but that wasn't my point at the time.)
I think this is exactly parallel to Kaj's observation. I think the advice "Learn to love failure" for entrepreneurs is getting at something similar, though my brain's too fuzzy right now to be quite sure.
This technique seems to apply in a whole lot of places. What's its most general statement? Something like this, but I'm not convinced I've got it down to its essence:
When there's some thing X that you trust, so that you aim to make your beliefs match X, beware that doing so may tend instead to distort your estimate of X to match your pre-existing beliefs. One way around this is to allow your beliefs to diverge from X a little, at least temporarily, so that you can figure out what X says without worrying about whether that matches your beliefs.
... At which point it strikes me that allowing your beliefs to diverge from X, when you're sure that X is very reliable, is in fact deliberate irrationality (and, note, not the same deliberate irrationality as Kaj is talking about: they're at different levels of meta-ness, as it were), and that perhaps we have here an example of when deliberate self-deception, or something like it, might help you.
Caused by: Purchase Fuzzies and Utilons Separately
As most readers will know by now, if you're donating to a charity, it doesn't make sense to spread your donations across several charities (assuming you're primarily trying to maximize the amount of good done). You'll want to pick the charity where your money does the most good, and then donate as much as possible to that one. Most readers will also be aware that this isn't intuitive to most people - many will instinctively try to spread their money across several different causes.
I'm spending part of my income on charity, too. Admittedly, this isn't much - 30 USD each month - but then neither is my income as a student. Previously I had been spreading that sum to three different charities, each of them getting an equal amount. On at least two different venues, people had (not always knowingly) tried to talk me out of it, and I did feel that their arguments were pretty strong. Still, I didn't change my ways, even though there was mental pressure building up, trying to push me in that direction. There were actually even some other charities I was considering also donating to, even though I knew I probably shouldn't.
Then I read Eliezer's Purchase Fuzzies and Utilons Separately. Here was a post saying, in essence, that it's okay to spend some of your money in what amounted to an irrational way. Yes, go ahead and spread your money, and go ahead and use some of it just to purchase warm fuzzies. You're just human, after all. Just try to make sure you still donate more to a utilon maximizer than to purchasing the fuzzies.
Here I was, with a post that allowed me to stop rationalizing reasons for why spreading money was good, and instead spread them because I was honestly selfish and just buying a good feeling. Now, I didn't need to worry about being irrational in having diversified donations. So since it was okay, I logged in to PayPal, cancelled the two monthly donations I had going to the other organizations, and tripled the amount of money that I was giving to the Institute Which Shall Not Be Named.
Not exactly the outcome one might have suspected.
A theme that has come up several times is that it's easier to lie to others if you believe in the lies yourself. Being in a community where rationality is highly valued, many people will probably want to avoid appearing irrational, lest they lose the respect of others. They want to signal rationality. One way to avoid admitting irrationality to others is by not admitting it to yourself. But then, if you never even admit your irrationality to yourself, you'll have a hard time of getting over it.
If, on the other hand, the community makes it clear that it's okay to be irrational, for as long as you're trying to get rid of that, then you can actually become more rational. You don't need to rationalize reasons why you're not being irrational, you can accept that you are irrational and then change it. Eliezer's post did that for me, for one particular irrationality [1]. So let me say that out loud: It's okay to be irrational, and to admit that. You are only a human.
Failing to realize this is probably a failure mode for many communities which try to extinguish a specific way of thinking. If you make a behavior seem like it's just outright bad, something which nobody should ever admit to, then you'll get a large amount of people who'll never admit to it - even when they should, in order to get over it.
And it's not just a community thing, it's also an individual thing. Don't simply make it clear to others that some irrationality is okay: make it also clear for yourself. It's okay to be irrational.
Footnote [1]: Note that Eliezer's post didn't extinguish the irrationality entirely. I'm still intending on using some of my money on purchasing warm fuzzies, once my total income is higher. But then I'll actually admit that that's what I'm doing, and treat my purchases of fuzzies and utilions as separate cases. And the utilon purchasing will be the one getting more money.