My recollection is that at least one study showed some regression to the mean in confidence -- highly-skilled people tended to underestimate themselves.
To the best of my knowledge, this sort of effect is mostly not an effect of underestimation, but rather of either misestimating the skills of others, and/or of restricting the group against which one evaluates oneself.
Generally, it seems, if one has no knowledge of others' skills in an area, an estimation of one's own skill level is likely wildly inaccurate; if one's knowledge of others' skills is biased in favor of a non-random group, one's own skill is likely closer to the mean of that group than expected.
That is to say, if you're better at X than your friends are, you're probably not as good as you think; if you're jealous of your friends' skill at X you're probably better than you think. On the other hand, if you measure your success against the best and brightest in a field, you're probably wildly underestimating yourself; but if you're proud of being good at something and compare yourself to random people you meet you're probably substantially overestimating yourself.
People posting on LW are strongly self-selected in favor of rationality, so anyone reading this is probably closer to average within this community than they think they are!
Drivers think they are comparing themselves to the other drivers they see on the road. In fact, they are comparing themselves to the drivers whose skill they have most cause to think about, which is overwhelmingly often the worst drivers on the road.
Consider the problem of an agent who is offered a chance to improve their epistemic rationality for a price. What is such an agent's optimal strategy?
A complete answer to this problem would involve a mathematical model to estimate the expected increase in utility associated with having more correct beliefs. I don't have a complete answer, but I'm pretty sure about one thing: From an instrumental rationalist's point of view, to always accept or always refuse such offers is downright irrational.
And now for the kicker: You might be such an agent.
One technique that humans can use to work towards epistemic rationality is to doubt themselves, since most people think they are above average in a wide variety of areas (and it's reasonable to assume that merit in at least some of these areas is normally distributed.) But having a negative explanatory style, which is one way to doubt yourself, has been linked with sickness and depression.
And the inverse is also true. Humans also seem to be rewarded for a certain set of beliefs: those that help them maintain a somewhat-good assessment of themselves. Having an optimistic explanatory style (in an nutshell, explaining good events in a way that makes you feel good, and explaining bad events in a way that doesn't make you feel bad) has been linked with success in sports, sales and school.
If you're unswayed by my empirical arguments, here's a theoretical one. If you're a human and you want to have correct beliefs, you must make a special effort to seek evidence that your beliefs are wrong. One of our known defects is our tendency to stick with our beliefs for too long. But if you do this successfully, you will become less certain and therefore less determined.
In some circumstances, it's good to be less determined. But in others, it's not. And to say that one should always look for disconfirming evidence, or that one should always avoid looking for disconfirming evidence, is idealogical according to the instrumental rationalist.
Who do you think is going to be more motivated to think about math: someone who feels it is their duty to become smarter, or a naive student who believes he or she has the answer to some mathematical problem and is only lacking a proof?
You rarely see a self-help book, entreprenuership guide, or personal development blog telling people how to be less confident. But that's what an advocate of rationalism does. The question is, do the benefits outweigh the costs?