I think it means "X is a mind-independent object that would and should be labelled Y by an onlooker speaking my language"

I think you've missed the part where that is still a label in your mind, being attached to a cluster of sensory experiences.

I believe there are stars and planets no one has ever seen, or had a chance to label as such, Don't you?

In such cases, the sensory experience clusters you're labeling are memories associated with the labels "star" and "planet".

However, this has little to do with an X-is-Y identity. In order to say "X is Y", there has to be an X and a Y, and you are speaking only here of the hypothesized existence of various X's that you would then label Y.

In any event, this and this are relevant here, in case you've missed them.