I don't like discussing general cases when I don't have some concrete examples. The only ones I can think of are boring cases of coercion involving unethical mindreaders.
Yes, I agree: the utility of having a belief only makes sense when for some reason you are rewarded for actually having the belief instead of acting as though you have the belief.
OK, since theism is unique in this aspect, in order to generalize away from the theistic, let's use the utility for acting-as-though-you-believe instead of the utility for actually believing, because in most cases, these should be the same.
... but then, as soon as you do this, the argument become just about choosing actions based on average expected utility and there's nothing c...
It has been claimed on this site that the fundamental question of rationality is "What do you believe, and why do you believe it?".
A good question it is, but I claim there is another of equal importance. I ask you, Less Wrong...
What are you doing?
And why are you doing it?