(Argh, see edit. Of course I forgot to include the actual argument.)

Well it's certainly a hint that there is something we are confused about. If you are talking about quantum many-worlds, then we can at least prevent our heads from exploding by using the notion of measure to talk about the probability of a world, which even applies to the supposedly uncountably many worlds. (I'm out of my depth here but I think we would then be talking about physical quantum amplitudes in configuration space, rather than subjective probabilities... which mysteriously correspond.)

If you are talking about a single spatially infinite universe... then yeah I don't know how to deal with that. Although I'd at least note that it is a very strange epistemological state that I am in, if I think that I somehow came to have accurate, meaningful beliefs about infinitely many, spatially infinite objects. How did that information get to me?