Imagine an agent that maximizes an altruistic utility function. It still only maximizes its own utility, no one else's. And its utility function wouldn't even directly depend on other agents' utility (you might or might not care about someone else's utility function, but a positive dependence on its value could cause a runaway feedback loop). But it does value other agents' health, happiness, freedom, etc. (i.e. most or all of the same inputs that would go into a selfish agent's utility function, except aggregated).

Two such agents don't have to have exactly the same utility function. As long as A values A's happiness, and B values A's happiness, then A and B can agree to take some action that makes A happier, even using ordinary causal decision theory with no precommitment mechanism.