I don't follow... Can you give an example where beliefs would matter?
I am not sure if I am answering your question, but:
a) if AI is trying to maximize X, and it has a possibility to do Y, then it matters whether AI believes that Y is X. For example an asteroid is going to hit the Earth, and it is not possible to completely avoid human deaths, which AI tries to avoid. But AI could scan all people and recreate them on another planet -- is this the best solution (all human lives saved) or the worst one (all humans killed, copies created later)?
b) it's not only about what AI believes, but human beliefs are also important, becau...
From Wolfgang Schwarz's Belief Dynamics Across Fission:
Also see: Sleeping Beauty problem.