The physical property to which you refer is deemed color only because it can induce an experience of color. And the experience of color can occur without that specific external stimulus. So the true nature of color must be sought within the brain.

Here is the confusion underlying this whole mess. There are three types of things which color can apply to: objects, light, and experiences. These are related causally: blue objects cause blue light which causes blue experiences; and evidentially: a blue experience is evidence that there was blue light, which is evidence that there was a blue object. However, color as it applies to objects, light, and experiences are three separate entities with different reductions. We use them interchangeably because the causal and evidential relationships allow them to substitute for eachother in almost all contexts.

If you start with one of blue objects, light, or experience clearly defined, then you can use that definition plus the causal/evidential relationships to define the other two. The natural way to define them is to define all three only in relation to eachother - ie, refer only to the entire structure, and depend on the ability to compare the color of reference objects/light/experiences to keep the definition stable. Fortunately, some discoveries from physics have enabled a simple physical description of blue light. Blue light is any light made predominately of photons with a wavelength close to 470nm. Based off that definition, a blue object is one that reflects or produces blue light, and a blue experience is one involving some particular set of neurons which I identify by their causal relationship to blue light. But stimulating these neurons without using light still makes a blue experience, and I could in principle identify those neurons some other way - for example, if I were to discover that protein X is found only in blue-experience neurons, then I could define a blue experience as an experience involving neurons containing protein X, and then define blue light and objects based on that.

There are some other strange entities which can have color because of causal relations, too. For example, the number 255 (#0000FF) is blue because it causes blue photons to be produced when written in the right part of a CSS file.