I talked to someone at an OB meetup who did change his mind about something important, by use of the full-fledged crisis of faith, with his use occasioned by Eliezer's post. It impacted his practical actions in significant ways. If each of us does one of these every three years, with something equally important, the technique will be paying off hugely.
For myself: I went to a coffee shop shortly after Eliezer's post, with a free afternoon and intent to try the technique. No particular subject matter. I started by making a list of "topics potentially warranting crises of faith", which itself probably made me more aware of existing gaps in my thinking. Then I picked the most emotionally difficult topic from my list, got a bit of the way in... and changed my mind about whether I wanted to think that one through. Then I picked another topic (also a practical, real-world matter) and... got partway through, farther than above, with some but not huge change to my beliefs and practices. I should try the procedure again.
Like Yvain, I'd love to hear others' experiences with the Crisis of Faith technique, positive or negative.
Since there's been much questioning of late over "What good is advanced rationality in the real world?", I'd like to remind everyone that it isn't all about post-doctoral-level reductionism.
In particular, as a technique that seems like it ought to be useful in the real world, I exhibit the highly advanced, difficult, multi-component Crisis of Faith aka Reacting To The Damn Evidence aka Actually Changing Your Mind.
Scanning through this post and the list of sub-posts at the bottom (EDIT: copied to below the fold) should certainly qualify it as "extreme rationality" or "advanced rationality" or "x-rationality" or "Bayescraft" or whatever you want to distinguish from "traditional rationality as passed down from Richard Feynman".
An actual sit-down-for-an-hour Crisis of Faith might be something you'd only use once or twice in every year or two, but on important occasions. And the components are often things that you could practice day in and day out, also to positive effect.
I think this is the strongest foot that I could put forward for "real-world" uses of my essays. (Anyone care to nominate an alternative?)
Below the fold, I copy and paste the list of components from the original post, so that we have them at hand: