That's how I used to phrase it, but now I would say instead that you switch what the things are mapped to. I think of the labels themselves as qualia, so that switching just the labels would be like switching the experience of "blue" with the experience of "red".