It's taken me a while, and this is an old post, but I think I've found what I've wanted to say:
Where does risk figure in all this? It almost sounds like you equate "optimal" with the path of zero risk, or at least the path of zero unknown risk.
Such an attitude towards risk would not be "optimal" if you wanted to say, play Elon Musk's game of experimental rocket technology.
And if we want to talk about "value", then a person who is willing to take on risk is very often viewed as more valuable.
Musk says:
I think I feel fear quite strongly. So it’s not as though I just have the absence of fear, I feel it quite strongly. But there are just times when something is important enough, you believe in it enough, that you do it in spite of fear. In starting SpaceX, I thought the odds of success were less than 10%, and I just accepted that actually, probably, I would just lose everything.
“But that maybe would make some progress, if we could just move the ball forward, even if we died, maybe some other company could pick up the baton and keep moving it forward; so that would still do some good. Yeah, same with Tesla. I thought the odds of a car company succeeding were extremely low. If you just accept the probabilities, then that diminishes fear.
“Like people shouldn’t think ‘Well I feel fear about this, and therefore I shouldn’t do it.’ It’s normal to feel fear. Like you’d have to … there’d have to be something mentally wrong if you didn’t feel fear.
In many areas, optimization is a process of failing forwards. A process of consequentialism, rather than utilitarianism.
Sometimes, nobody has the information that we need in order to optimize. We simply need to risk experimenting and finding out that info for ourselves.
...
...
Note, when speaking of risk, I don't only mean big things. I more generally mean - proceeding even when there is a space of uncertainty.
In other questions, I've noticed you've covered what is called "ask vs guess culture". When we have to ask someone before we can do something, often what bothers us is not the asking, but the potential of hearing a no.
But is hearing a no such a bad thing? Is having to negotiate a boundary such a bad thing? I think one often too strongly assumes that people will answer no.
If one looks into "rejection therapy", there are many videos of someone experimenting with asking people for things - like, the person goes to a store that cuts and washes the hair of dogs and cats, and he asks the people at the counter if they can cut his hair.
They take it in great humor and laugh, and then say no.
In another, the person asks a stranger if they can play soccer in the stranger's backyard. This complete stranger says yes!
The person continues to ask many weird and wonderful things, and gets a variety of different rejections and approvals. And I'd say that this process of rejection therapy was one of the person "optimizing" their own sense of becoming comfortable with asking for what is wanted, and comfortable with the potential for rejection.
A recent book, "The Courage To Be Disliked" covers a very similar thing.
To have the courage to take risks, and become comfortable with rejection and boundaries is to have the courage to be one's own separate self, with one's own separate boundaries.
Freedom is not only being free from the boundaries of others.
To be free also means to be able to deploy and retract one's own boundaries. Even when there is a risk that others do not like them.