One of the key questions that we all face is to figure out a purpose in life, a direction, a goal. However, that is not an easy question. In fact, it's not even easy to say what kind of thing human values are in general. Our brain is fragmented in so many different ways: past, present and future preferences; emotional, intuitive and cognitive systems and multiple layers of meta-preferences. Given the tangle of confusion and that a solution that seems right for me might not seem right for you, I would suggest that finding a pragmatic approach to this problem might be even more important than actually trying to solve it.
I would propose that contrary to current rationalist wisdom trying to pull some kind of consistent utility function out of this can be counterproductive. I've honestly burned up far too many brain cycles trying to do this; sometimes there is value in just doing something and forgetting about optimality. After all, utility pumps are quite rare and people tend to catch on when they are being pumped anyway.
Consider the following: What should we optimise for personal utility or our values? Assume that we include the utility we gain from achieving our values in personal utility. If you believe in moral realism, then you have an obvious reason to pursue your values even when it doesn't benefit you, but what about otherwise? Should you take the self-centered approach of only caring about your values insofar as they seem likely to provide you with utility?
Your hedonic component (or components)would be quite satisfied with this solution, but the part of you that has values outside of yourself would not be. Each part self-affirms its own viewpoint. If we have no real resolution about which part deserves precedent, then a sensible default would be to assign value to both.
This gives us a reason to move past pure hedonism (or hedonism + values as instrumental for hedons), but do we have a reason to go any further? After all, there's a significant difference between merely attempting to realise your other-directed values and being deeply committed to achieving them.
Maybe we don't have any reason from a principled perspective, so I suppose we'll now have to fallback to the instrumental (and admittedly self-directed) perspective. Firstly, if we aren't committed to a goal, we'll be unlikely to achieve it even when we easily could have, we won't value success and even small efforts are likely to be draining. Making a lukewarm effort may seem like a natural response to this uncertainty, but for these reasons it is usually a terrible deal. Secondly, the ups and downs of life are such that we are almost guaranteed to have periods where our experience is terrible. If we have some kind of purpose, then we'll at least have something to hold onto, some way of ensuring that our internal narrative doesn't just generate more suffering for ourselves (It also reduces risk: https://markmanson.net/diversify-your-identity). Thirdly, we avoid the nihilism or detachment that are incredibly damaging for most people's psyche. Again, lukewarm goals don't help here as they'll feel clearly purposeless.
Given that we have all of these instrumental arguments, why all the fuss about producing a non-instrumental argument first? Even if much of the motivation might end up being from these instrumental arguments, I think that it is important that not all of it is. If that were the case, then I suspect that pursuing the goals would likely end up feeling pointless (pursuing a goal for the purpose of having a goal) or disingenuous. In other words, the instrumental reasons by themselves don't necessarily deliver the instrumental benefits without at least some non-instrumental component.
I think that phenomenologically, you're right. Other-directed goals (need for relatedness, in SDT terminology) feel like they're essentially other-directed.
I think that the evolutionary cause for having other-directed goals is directed at your own genetic proliferation, and I also think that autonomously holding other-directed goals improves your own well-being, even above and beyond the benefits you get because they like you for it. Eg. Gore et al. 2009.
Stated differently, even if you're optimising completely selfishly, you'll have to be unselfish. We care about others simply because they are important to us, not because they make us happy. They are a terminal value. If they are instrumental, we don't get the benefits to well-being. But caring for them terminally also carries benefits to ourselves. I think that's wonderful!