On the other hand, if I don't have enough information even given infinite mathematical power, it implies that the world must contain copies of me that will see different coinflip outcomes (if there was just one copy, mathematics would be able to pin it down)

I don't get this implication. Are you suppressing some premises here? I am sympathetic to the idea that all non-logical uncertainty is best thought of as indexical, but I usually think about this as indexical uncertainty about which of various logically possible worlds I live in, not indexical uncertainty about who I am in this world.

As an exception to your dichotomy, consider uncertainty about the laws of nature. I very much doubt we actually possess enough information about the world so that if we had infinite mathematical power we could logically deduce the correct laws of nature from our current evidence (although maybe you believe this, in which case, would you also believe it for people in the Stone Age?), but it also doesn't seem right that there are copies of me in this world living in environments with different laws of nature, at least not if we mean something sufficiently fundamental by "law of nature".

Maybe by "the world" you mean a Tegmark Level IV multiverse? If that's the case, it's probably worth making clear, since that's definitely not the usual sense of the word.