Normatively, seeking disconfirmation and not finding it should make you more certain. And if you do become less certain, I'm not convinced this necessarily makes you less determined – why couldn't it heighten your curiosity, or (especially if you have something to protect) make you more determined to try harder and return, with justification, to the same certainty?

I'm beginning to suspect that providing theoretical justifications for inherently irrational humans is a waste of time. All I can say is that my empirical evidence still holds, and I observe this in myself. Believing that I'm going to fail demoralizes me. It doesn't energize me. I'd love to have my emotions wired the way you describe.

Or, how about the student who believes they may have the answer, and has a burning itch to know whether this is the case? Or the one with something to protect?

The point is that the student benefits from believing something regardless of its truth value. The greater the extent to which they believe their idea is valid, the more thinking about math they'll do.

While I'm not very familiar with these literatures, I suspect encouraged overconfidence is often just a motivational hack, in which case you should again look for a Third Alternative: can you find the willpower to do this, while having properly calibrated belief in its success? Alternately, it might hack around people not realizing (even deliberatively) the potential payoff of success and/or the upside of failure; but then you should determine and appreciate these things.

I'd love to hear your Third Alternative.

Also, y'know, there could be some issues on which many people really are underconfident.

It seems to happen occasionally.