So only by knowing about both dilemmas can you define value. Since, by the problem definition, you only know of one of them, defining relative value is impossible.

You'd only need to know what sort of utility function for that. Sure, you wouldn't know that the other deal like that is happening right there, so it might not be possible to reach counterfactual agreement in this particular case, but as a general rule, it does seem possible to outperform agents that don't go for this sort of "I take a hit for you if you'd take a hit for me in this sort of scenario where stakes are reversed". Which leaves multiple questions open, I agree.