Obviously, it's epistemically good to change your mind after receiving better information... but there also seems to be a tendency to cling to our opinions. It seems that 'being wrong' negatively affects your self-image, status. Even if we (as aspiring rationalists) can overcome such biases, how to deal with that in a sub-rational world which sees things differently?
People seem to like people who are very confident about their ideas, rather than people that change their mind, even for apparently good reasons.
I noticed there is certain comic effect to immediately matter-of-factly agreeing with arguments on the side of your opponent's position that are correct, but were picked as debate-soldiers, expected to be fought or in some way excused/objected to. :-)
There are many pleasant benefits of improved rationality:
I'd like to mention two other benefits of rationality that arise when working with other rationalists, which I've noticed since moving to Berkeley to work with Singularity Institute (first as an intern, then as a staff member).
The first is the comfort of knowing that people you work with agree on literally hundreds of norms and values relevant to decision-making: the laws of logic and probability theory, the recommendations of cognitive science for judgment and decision-making, the values of broad consequentialism and x-risk reduction, etc. When I walk into a decision-making meeting with Eliezer Yudkowsky or Anna Salamon or Louie Helm, I notice I'm more relaxed than when I walk into a meeting with most people. I know that we're operating on Crocker's rules, that we all want to make the decisions that will most reduce existential risk, and that we agree on how we should go about making such a decision.
The second pleasure, related to the first, is the extremely common result of reaching Aumann agreement after initially disagreeing. Having worked closely with Anna on both the rationality minicamp and a forthcoming article on intelligence explosion, we've had many opportunities to Aumann on things. We start by disagreeing on X. Then we reduce knowledge asymmetry about X. Then we share additional arguments for multiple potential conclusions about X. Then we both update from our initial impressions, also taking into account the other's updated opinion. In the end, we almost always agree on a final judgment or decision about X. And it's not that we agree to disagree and just move forward with one of our judgments. We actually both agree on what the most probably correct judgment is. I've had this experience literally hundreds of times with Anna alone.
Being more rational is a pleasure. Being rational in the company of other rationalists is even better. Forget not the good news of situationist psychology.