Right, in degenerate cases, when there's nothing to be learned, the two extremes of learning nothing and everything coincide.

In the case where your prior says "the past is not informative about the future". You learn nothing. A degenerate prior, not degenerate situation.

To the extent that I understand your navigational metaphor, I disagree with this statement. Would you kindly explain?

Imagine a bowl of jellybeans. you put in ten red and ten white. You take out 3, all of which are red, the probability of getting a red on the next draw is 7/17.

Take another boal, have a monkey toss in red beans and white beans with 50% probability. You draw 3 red, the draw probability is now 50% (becuase you had a maxentropy prior).

Take another boal. Beans were loaded in with unknown probabilitities. You draw 3 red, your draw probability is 4/5 red.

See how depening on your assumptions, you learn in different directions with the same observations? Hence you can learn in the wrong direction with a bad prior.

Learning sideways is a bit of metaphor-stretching, but if you like you can imagine observing 3 red beans proves the existence of god under some prior.

given that prior and the agent's observations

Yes yes. I was being pedantic because your post didn't talk about priors and inductive bias.

very little

where part of the change is revealed to you through new observations, you have to keep pace.

I thought of that. I didn't think enough. "very little" was the wrong phrasing. It's not that you do less updating, it's that your updates are on concrete things like "who took the cookies" instead of "does gravity go as the squre or the cube" because your prior already encodes correct physics. Very little updating on physics.