The human experience of colour is not really about recognizing a specific wavelength of light.

True, but irrelevant to the subject at hand.

the qualia of colour are associated more with the invariant surface properties of objects than they are with invariant wavelengths of light.

No, the qualia of color have nothing to do with the observed object. This is the pons asinorum of qualia. The experience of color is a product of the invariant surface properties of objects; the qualia of color is a product of the relationship between that experience and other similar experiences.

A human looking at an optical illusion might say, "That looks red, but it's really white," acknowledging that spectral color is objective, but psychophysical color is more malleable. But compare that sentence to "that sounds good, but it's really bad." Statements about color aren't entirely subjective--to some extent they're about fact, not opinion.

Statements about qualia are about the subjective aspect of an experience: e.g., red is the color of rage; of love; the color that means 'stop.'