Suppose that my husband and I believe that while we're sleeping, someone will paint a blue dot on either my forehead, or my husband's, determined randomly. We expect to see a blue dot when we wake up... and we also expect not to see a blue dot when we wake up. This is a perfectly reasonable state for two people to be in, and not at all problematic.

Suppose I believe that while I'm sleeping, a powerful AI will duplicate me (if you like, in such a way that both duplicates experience computational continuity with the original) and paint a blue dot on one duplicate's forehead. When I wake up, I expect to see a blue dot when I wake up... and I also expect not to see a blue dot when I wake up. This is a perfectly reasonable state for a duplicated person to be in, and not at all problematic.

Similarly, I both expect to experience bowling with you, and expect to not experience bowling with you (supposing that the original continues to operate while the simulation goes bowling).