a truthful deity told you that the mean was 3.5

I think I'd answer, "the mean of what?" ;)

I'm not really qualified to comment on the methodological issues since I have yet to work through the formal meaning of "maximum entropy" approaches. What I know at this stage is the general argument for justifying priors, i.e. that they should in some manner reflect your actual state of knowledge (or uncertainty), rather than be tainted by preconceptions.

If you appeal to intuitions involving a particular physical object (a die) and simultaneously pick a particular mathematical object (the uniform prior) without making a solid case that the latter is our best representation the former, I won't be overly surprised at some apparently absurd result.

It's not clear to me for instance what we take a "possibly biased die" to be. Suppose I have a model that a cubic die is made biased by injecting a very small but very dense object at a particular (x,y,z) coordinate in a cubic volume. Now I can reason based on a prior distribution for (x,y,z) and what probability theory can possibly tell me about the posterior distribution, given a number of throws with a certain mean.

Now a six-sided die is normally symmetrical in such a way that 3 and 4 are on opposite sides, and I'm having trouble even seeing how a die could be biased "towards 3 and 4" under such conditions. Which means a prior which makes that a more likely outcome than a fair die should probably be ruled out by our formalization - or we should also model our uncertainty over how which faces have which numbers.