That someone knows every physical fact about gold doesn't make that person own any gold.

That's not the point: the point is why is it necessary, in the case of experience, and only in the case of experience to instantiate it in order to fully understand it. Obviously, it is true a that a descirption of a brain state won't put you into that brain state. But that doesn't show that there is nothing unusual about qualia. The problem is that there in no other case does it seem necessary to instantiate a brain state in otder to undertstand something.

If another version of Mary were shut up to learn everything about, say, nuclear fusion, the question "would she actually know about nuclear fusion" could only be answered "yes, of course....didn't you just say she knows everything"? The idea that she would have to instantiate a fusion reaction within her own body. That doesn't apply, any more than a description of photosynthesis will make you photosynthesise. We expect that the description of photosynthesis is complete, so that actually being able to photosynthesise would not add anything to our knowledge.

The list of things which the standard Mary's Room intution doesn't apply to is a long one. There seem to be some edge cases.: for instance, would an alternative Mary know everything about heart attacks without having one herself? Well, she would know everything except what a heart attack feels like —and what it feels like is a quale. the edge cases, like that one, are cases are just cases where an element of knowledge-by-acquaintance is needed for complete knowledge. Even other mental phenomena don't suffer from this peculiarity. Thoughts and memories are straightforwardly expressible in words — so long as they don't involve qualia.

So: is the response "well, she has never actually instantiated colour vision in her own brain" one that lays to rest and the challenge posed by the Knowledge argument, leaving physicalism undisturbed? The fact that these physicalists feel it would be in some way necessary to instantiate colour means they subscribe to the idea that there is something epistemically unique about qualia/experience, even if it resists the idea that qualia are metaphysically unique.

Is the assumtion of epistemological uniqueness to be expected given physicalism? Some argue that no matter how much you know about something "from the outside", you quite naturally wouldn't be expected to understand it from the inside. However, if physicalism is taken as the claim that everything ultimately has a possible physical explanation, that implies that everything has a description in 3rd person, objective language — that everything reduces to the 3rd person and the objective. What that means is that there can be no irreducible subjectivity: whilst brains may be able to generate subjective views, they must be utlimately reducible to objectivity along with everything else. Since Mary knows everything about how brains work, she must know how the trick is pulled off: she must be able to understand how and why and what kind of (apparent) subjetivity is produced by brains. So the Assumption of Epistemelogical Uniqueness does not cleanly rescue physicalism, for all that it is put forward by physcialists as something that is "just obvious".