I have a question and I think maybe your answer will make it easier for other people to understand what you are arguing.

What about people who are color blind? They see for instance red where in objective reality the objects wavelength is "green". What happens here in your view? In the persons experience he still see red, but it should be green... And eventhough we know approximately the processes that do that people are color blind, this seems to be a interesting question in your model.