Sensitivity to priors is the same as sensitivity to new evidence. And when we're sensitive to new evidence, our estimates are likely to change, which is another reason they're uncertain.

The reason this phenomena occurs is because we are uncertain about some fundamental frequency, or a model more complex than a simple frequency model, and probability(heads|frequency of heads is x)=x.