I assumed we couldn't observe our copies, because if we could, then they'd be observing them too. In other words, somebody's experience of observing a copy would have to be fake - just a view of their present reality and not of a distinct copy.

This all follows from the setup, where there can be no difference between a copy (+ its environment) and the original. It's hard to think about what value that has.