"So what an expert rationalist should do to avoid this overconfidence trap?"
You mean, how should one overcome bias? Be less wrong, if you will? You've come to the right place. David Balan had a post that didn't receive enough attention over at OB: http://www.overcomingbias.com/2008/09/correcting-bias.html This comment roughly paraphrases the points I made there.
If we can identify a bias, presumably we can also identify the optimal outcome that would happen in the absence of such bias. There are two ways to achieve this, and I will post them in separate comments so they can be voted down separately.
If you can identify an optimal outcome for a situation in which you are likely to be biased, you can constrain yourself ahead of time such that you can't give in to the bias. The classical example is Odysseus tying himself to the mast to avoid giving in to the sirens' song. Tie yourself to the mast at a rational moment, so you don't err in a biased one.
Applying this to your example, if you are indeed trying to maximize returns on a portfolio, the last place you should make buy/sell decisions is on a loud, testosterone-laden trading floor. It's better to decide ahead of time on a model based on which one is going to manage: "If a tech stock has x earnings but y insider ownership, then buy." Stick to your rules [perhaps bind yourself through some automatic limit that you cannot circumvent] as long as they seem to be tending to achieve your goal [maximize return]. Revisit them if they don't seem to - but again, revisit them at a time you are not likely to be biased.
Agreed.
The classical example is Odysseus tying himself to the mast to avoid giving in to the sirens' song
You just stole that from the banner at OB
“Everyone complains of his memory, but nobody of his judgment." This maxim of La Rochefoucauld rings as true today as it did back in the XVIIth century. People tend overestimate their reasoning abilities even when this overconfidence has a direct monetary cost. For instance, multiple studies have shown that investors who are more confident of their ability to beat the market receive lower returns on their investments. This overconfidence penalty applies even to the supposed experts, such as fund managers.
So what an expert rationalist should do to avoid this overconfidence trap? The seeming answer is that we should rely less on our own reasoning and more on the “wisdom of the crowds”. To a certain extent this is already achieved by the society pressure to conform, which acts as an internal policeman in our minds. Yet those of us who deem themselves not very susceptible to such pressures (overconfidence, here we go again) might need to shift their views even further.
I invite you now to experiment on how this will work in practice. Quite a few of the recent posts and comments were speaking with derision about religion and the supernatural phenomena in general. Did the authors of these comments fully consider the fact that the existence of God is firmly believed by the majority? Or that this belief is not restricted to the uneducated but shared by many famous scientists, including Newton and Einstein? Would they be willing to shift their views to accommodate the chance that their own reasoning powers are insufficient to get the right answer?
Let the stone throwing begin.