Aumann's agreement theorem prevents that from happening to ideal epistemic rationalists; there's no analogue for instrumental rationality.

But...

Aumann's agreement theorem assumes common priors, what I described can only happen to instrumental rationalists with different utility functions. So the question is why we expect all rationalists to use One True Prior (e.g. Solomonoff induction) but each to use their own utility function.