What dire consequences should we expect if we do, in fact, deny that there is anything that is blue ?

For my money, the discussion in p.375 and onwards of Consciousness Explained says all there is to say (in addition to theories of electromagnetism, optics and so on) about the experience of color.

I can't really do justice to that section in a comment here, but I will note its starting point:

Many have noticed that it is curiously difficult to say just what properties of things in the world colors could be. [...] What is beyond dispute is that there is no simple, nondisjunctive property of surfaces such that all and only the surfaces with that property are red.

The key insight for me is here:

The fact that apples have the surface reflectance properties they do is as much a function of the photopigments that were available to be harnessed in the cone cells in the eyes of fructivores as it is of the effects of the interactions between sugar and other compounds in the chemistry of the fruit.

There is no reason, prior to megayears of evolution, to expect that anything such as color exists. That changes with apple trees' "need" to advertise the ripeness of their fruit, to creatures which, though hardly conscious, happen to be equipped with the ability to discriminate certain properties of the fruit at a distance. This "need" results from a) the existence of a certain optimization algorithm, Darwinian evolution, and b) contigent facts about the environment in which this algorithm unfolds.

What I take the "experience of color" to be, if it has to be anything, is an evolved equilibrium between competing strategies, together with the entire history of the genes in which these strategies were encoded.