It would be helpful to start this sort of discussion with a working definition of consciousness, as the term has various meanings in various contexts and still carries metaphysical connotations.

If one's consciousness suddenly became a totally different one, we know of no quantum particles that would change.

If we take the viewpoint that consciousness is a function of the brain's internal state, then swapping one consciousness with another should correspond to the appropriate change inside the brain.

Furthermore, swapping consciousnesses would make no changes to what is perceived. E.g. if one agent perceives p and time t and p' at the next time t+1, and another agent perceives q at time t and q' at time t+1, then if their consciousnesses are "swapped," the percepts would still be identical: p and q will be perceived at time t, and p' and q' will be perceived at t+1.

Observing an external event creates a change in the brain state that depends both on the actual event and on the current brain state. Thus, two people can observe the same event and come away with completely different perceptions (politics provides an illustrative example). If a change in consciousness implies a change in brain state, then we would not expect the brain state after the observation to be the same in both cases.