When I look out the window right now, I see a blue patch of sky. Am I seeing neurological associations?

No, the neurological associations are the act of seeing. Different receptors in your eyes are excited by various frequencies of light, and the strength and pattern of their activation are associated in various ways by your central nervous system. There is a mathematical description of all of the steps in that process, including its representation in your memory.

Color isn't "out there", which is why very different frequency combinations can be perceived as the same color.

You seem to be ascribing something ineffable to your sensation of color, and then proclaiming that it can't be comprehended; I fail to see how any study of the mathematics of quantum mechanics could convince you it's responsible for your supposed sensations. Good luck with that, I guess.