Often, people think about their self-worth/self-confidence/self-esteem/self-efficacy/self-worth in ways which seem really strange from a simplistic decision-theoretic perspective. (I'm going to treat all those terms as synonyms, but, feel free to differentiate between them as you see fit!) Why might you "need confidence" in order to try something, even when it is obviously your best bet? Why might you constantly worry that you're "not good enough" (seemingly no matter how good you become)? Why do people especially suffer from this when they see others who are (in some way) much better than them, even when there is clearly no threat to their personal livelihood? Why might you think about killing yourself due to feeling worthless? (Is there an evo-psych explanation that makes sense, given how contrary it seems to survival of the fittest?)
There might be a lot of diverse explanations for the diverse phenomena. I think providing more examples of puzzling phenomena is an equally valuable way to answer (though maybe those should be a comment rather than an answer?).
This seems connected to the puzzling way people constantly seem to want to believe good things (even contrary to evidence) in order to feel good, and fear failure even when the alternative is not trying & essentially failing automatically.
Some sketchy partial explanations to start with:
- Maybe there is a sense in which we manage the news constantly. It could be that we have a mental architecture which looks a lot like a model-free RL agent connected up to a world model, being rewarded for taking actions which increase expected value according to the world-model. The model-free RL will fool the world-model where it can, but this will be ineffective in any case where the world-model understands such manipulation. So things basically even out to rational behavior, but there's always some self-delusion going on at the fringes. (This only has to do with the observation that people sometimes try to make themselves feel better by finding arguments/activities which boost self-esteem, not with other weird aspects of self-esteem.)
- There's a theory that, in order to be trustworthy bargaining partners, people evolved to feel guilty/shameful when they violate trust. You can tell who feels more guilt/shame after some interaction with them, and you can expect these people to violate trust less often since it is more costly for them. Therefore feelings of guilt/shame can be an advantage. Self-worth may be connected to how this is implemented internally. So, according to this theory, low self-worth is all about self-punishment.
- Previously, I thought that self-worth was like an estimate of how valuable you are to your peers, which serves as an estimate of what resources you can bargain for (or, how strong of a bid can you successfully make for the group to do what you want) and how likely you are to be thrown out of the coalition.
- Now I think there's an extra dimension which has to do with simpler dominance-hierarchy behavior. Many animals have dominance hierarchies; humans have more complicated coordination strategies which involve a lot of other factors, but still display very classic dominance-hierarchy behavior sometimes. In a dominance-hierarchy system, it just makes sense to carry around a little number in your head which says how great (/terrible) a person you are, and engage in a lot of varying behaviors depending on your place in the hierarchy. Someone who is low in the hierarchy has to walk with their tail between their legs, metaphorically, which means displaying caution and deference. Maybe you have trouble talking to people because you need to show fear to your superiors.
So let me take my current working detailed model of how human minds work and see what it says about self-esteem, a phenomenon I'm fairly familiar with as I've done some work on it in the past and was successful such that I used to have self-esteem "problems" and now I don't, and the problems I do have that look sort of like issues of self-esteem have deeper roots than what is normally addressed in self-esteem self-help and positive psychology literature from what I've read.
To the extent that we can model human minds as hierarchies of control systems that aim to minimize prediction error and maintain various set points (presumably for adaptive reasons), we'd expect it to look like humans, using our ability to observe the behavior of others and introspect, are both trying to make correct predictions about how much esteem (seemingly a metric that tracts an important generator of status, prestige, and generalize capability to get things done) they have and maintain an amount of esteem necessary to fulfilling other preferences (i.e. minimizing the error in other predictions and maintaining other set points). This will result in a few different behaviors depending on conditions, remembering that in this model humans do thing by predicting that they will to cause the necessary neurons to fire to cause the predicted behaviors to happen:
Other outcomes seem possible, those are just the situations that were salient to me.
This also suggests a way to fix self-esteem, which I think matches most of what I've read about it: find a way to bootstrap esteem generation such that esteem error prediction is reduced and esteem starts moving up towards the set point, and specifically adopt a belief seeing you are more esteemed than you thought is part of why you are worthing of esteem such that it causes a positive feedback loop that takes you up to and maybe even past your set point.
On the other side, it also matches what we hear about imposter syndrome: find evidence that you are not in error so that you stop being surprised and maybe also move your set point up to its new value.
Finally, it suggests a way to avoid all this suffering over esteem: learn not to have a fixed esteem set point such that you can only make prediction errors, not feel bad that your prediction is also not predicting the set point.
Oh, to your other question, I have no special resources to recommend. I mostly arrived at a version of this theory on my own, then found out about Friston and perceptual control theory and was like "ah, great, someone already worked this out, one less thing for me to worry about!".
In my recent post on value drift I had a paragraph with a bunch of links, though I think they are the things you may have already seen. For completeness and in case you missed any of the ones I liked, here it is copied into this comment:
... (read more)