Not sure if I understood your hypothesis correctly: are we underconfident when there is a risk of experiment and consequent public failure, and overconfident when the ability cannot be easily measured?
This seems like it could explain why people have strong opinions... but refuse to bet on them. When betting becomes a realistic option, our instincts turn off the overconfidence and turn on the underconfidence.
Yes, and the idea is that the lifelong practice is to become neither overconfident nor under-confident, but rather to identify the point of calibration when evaluating ourselves, and keep track of it as it changes over time, and that tracking it actually helps us learn and grow.
I agree – betting introduces a very interesting dynamic in this process. It can create a shambles if one's confidence swings over and under the mark wildly, since people tend to overcorrect.
I have this weird experience - not sure that I describe it correctly - that some people have described me as "simultaneously the most humble and the most arrogant person they met". From my perspective, if I believe that I am good at something, I say it with confidence, and if I believe that I am bad at something, I admit it frankly. This seems to confuse some people for some reason.
There are multiple possible explanations: maybe my behavior is less sane than I believe, maybe I use weird reference groups, but maybe... people expect consistent confidence that is mostly independent of actual skills. That is, their usual experience is probably something like "high-status people are always overconfident, low-status people are always underconfident", and meeting someone who says "I am great at X, but I really suck at Y" is weird. In other words, other people do not really expect your confidence to be closely related to your skills. (But as I said, other explanations are also possible.)
There is the point that 80% of people can say that they are better than average drivers and actually be correct. People value different things in driving, and optimise for those things. One person’s good driver may be safe, someone else may value speed. So both can say truthfully and correctly that they are a better driver than the other. When you ask them about racing it narrows the question to something more specific.
You can expand that to social hierarchies too. There isn’t one hierarchy, there are many based on different values. So I can feel high status at being a great musician while someone else can feel high status at earning a lot, and we can both be right.
The concept of “illusory superiority” used to just be a punchline about human nature. Eighty percent of people believe they are above-average drivers, above-average lovers, and above-average vegetable choppers. Ah, humans: so charmingly irrational.
Then recently, independent sociology researcher Spencer Greenberg showed that illusory superiority is complemented by an equal but opposite effect: illusory inferiority. When he asked people to rate themselves on hundreds of competencies, he found that while most people believe they are above-average drivers, most people simultaneously believe they would be worse than average at driving a race car. When the bar is sufficiently raised, it seems that under-confidence becomes the norm.
Rob Reid discussed Spencer’s findings in a recent After On podcast about pandemic detection using search data. Rob and his guest seemed to agree that Spencer’s findings support the idea that maybe we should be more optimistic about human nature – perhaps we’re not just a bunch of overconfident apes!
But this optimism misses the far more interesting question: what should we actually do with this new information? Just like Taleb’s concept of antifragility expanded the definitional range of the spectrum of fragility beyond just “robust” to “delicate,” the empirical evidence for illusory inferiority helps us see more of the range of the calibration scale of self-assessment, from systematically biased overconfidence to systematically biased under-confidence. Understanding it better can help illuminate a path toward good calibration at the center. ( For background reading on calibration, see Probabilistic forecasts, calibration and sharpness by Gneiting.)
Why should we want to be well-calibrated in our judgements about our own abilities?
Being well-calibrated about your own abilities, i.e. neither over- nor under-confident, is not just about increasing self-knowledge. It is also a prerequisite for learning.
If you’re overconfident about your own abilities in a particular area, then there is no pressing need to spend time learning about it. If I believe I’m an amazing driver, why heed the annoyed honks directed at me? It’s clearly their problem. While feedback blindness is harder to pull off in strictly quantitative endeavors, like math and video games, it’s pretty easy to fall into it in subjective fields with slow or ambiguous feedback cycles. (Another complication that’s beyond the scope of this piece but worth mentioning: in highly social interactions, such as sales and pulpit-pounding, overconfidence itself can become a kind of social competence – independent of other types of competencies – which can lead to echo-chamber feedback loops.)
However, there are often good, highly adaptive reasons for deviating from calibration and becoming over- or underconfident about your own relative standing compared to others. Let’s look at everyday competencies outside of the realm of highly quantitative feedback–things like driving, cooking, choosing a house, or a course of study. In these ambiguous, subjective contexts, grading yourself “higher than average” has basically no immediate social consequences, and your assessment is unlikely to ever be directly put to the test (in some cases it would be very difficult to measure). In other words, your own judgment about your relative rank in these types of scenarios is unlikely to be questioned. Cockiness then, is likelier to have positive, protective effects in this situation than to produce negative feedback.
Neurobiology tells us that being ranked lower on the social hierarchy ladder is incredibly stressful: Adult baboons exhibit higher glucocorticoid levels as adults if they have a subordinate mom. (What is a subordinate mom? One who gets out of the way of the dominant females of the troop, eats less and later, and doesn’t respond in kind to direct aggression.) Being low status is really stressful, and has long-term effects, so it makes sense to come up with ways to avoid this stress.
In ambiguous, subjective contexts, grading yourself “high” protects against this stress. Believing you are an above-average driver has no social consequences, and will never be tested, so why not avoid marinating your amygdala in stress hormones?
So, that may explain (some) overconfidence. But what about under-confidence? In strict or quantitative contexts, there is a greater threat that must be guarded against that takes precedence over the long-term chronic stress of social subordination: the acute stress of public humiliation. When comparing yourself to a professional race car driver, it may be better to underestimate your capabilities, or else risk being publicly shown up, considered low-trust, or worse, potentially even getting hurt or hurting others.
A final point, we often, to our detriment, mix up subjective rankings, which are socially constructed, with those that are empirically measurable.
Our highly sensitive primate sense of hierarchy awareness evolved a long time ago and probably doesn’t do a great job distinguishing between the two.
Empirical rankings certainly have their place – I think the world would be less bright without the Olympics, 4th grade spelling bees, and Dota 2 tournaments. We just get into trouble when we take socially constructed rankings – best pie at the County Fair, most valuable player on the team, the Nobel Prize – and confuse the two kinds of recognition and rewards. We need to be intellectually honest about the difference, and weigh them accordingly when considering our own self-image. Empirical performance measures are excellent learning tools, though they can be harsh tutors. Social recognition gives us a different kind of information, and a different kind of value: are we contributing to our communities in a way that truly connects? That usually feels good to all parties involved – except for the last kid to get picked for the dodgeball game. This is the downside of socially constructed rewards.
We have specific, adaptive cognitive biases that pull us away from calibration, and simply drawing conscious attention to them can help our brains unlearn the patterns we fall into when the amygdala (the fear center of our brains) is in the driver’s seat.You want to have the ability to take on big, challenging, exciting new things, even if it's a little bit uncomfortable.
Being socially calibrated is a keystone life skill if you want to excel at learning, and at managing your own psychology along the way. Knowing when you need to learn more, and when you shouldn't be too confident, just makes life’s journey a smoother ride, and is more likely to endear you to your fellow travelers along the way, especially the ones you’d like to learn from.