Interestingly, I don't think he bluntly said the (perhaps) most fundamental part of his position: what the color red looks like is a fact about Mary's brain, not about the color red. This also explains how qualia are essentially incommunicable: I don't have Mary's brain, so I can't really observe what it is like for information to be processed in that brain.
That doesn't explain why qualia are incommunciable, since others stuff that happens in brains is communicale. We expect Mary to understand everything except qualia.
They are incommunicable because you cannot occupy any arbitrary brainstate you want to. See the ability hypothesis.
http://esr.ibiblio.org/?p=1192#more-1192
ADDED: Even if you disagree with ESR's take, and many will, this is the clearest definition I have seen on what qualia is. So it should present a useful starting point, even for those who strongly disagree, to argue from.