They are not the same, but our visual system is trying to tell us they are the same.

Yeah, this makes sense. The trick is asking, "The same what?" The answer, "Color" is not descriptive enough.

I never meant to say it is right or wrong for the visual system to do whatever it is doing. I mean to say it is wrong to expect something different from the visual system than what it is doing.