Qualia are neither a barrier ("sense data") to knowledge of the physical world, nor are they illusory. Rather, qualia are a constant concomitant to perception of the physical world. The best hypotheses to explain our experience involve positing both qualia and the rocks, trees, and animals that the "sense-data" of certain philosophers would screen off.

Try this experiment. Set an apple on the table and look at it. Now approach closer and keep looking at it.

Your experience changes during this experiment. However, the apple doesn't seem to change, the table doesn't seem to change, etc. Your experience involves more redness, but the apple hasn't gotten any redder, nor seemed to. Some of our experience amounts to cognition or alleged cognition of the external world - the apple is red, it is sitting on the table, etc. But some does not - it is just an internal aspect of the way we sense the world. Philosophers call the latter "qualia".

Drugs or sleep deprivation or psychosis may cause someone to hallucinate an apple. If they have enough relevant history, they may correctly doubt that the apparent apple really exists. But they won't doubt that they are having an experience. What does that experience involve? Qualia, but no external object. If the hallucinator says "I hope the apple is real, because I'm hungry" the word "apple" in that sentence does not refer to the qualia. (Nor does "the apple" said by someone having a veridical experience.) It does not refer at all.

Armchair philosophy cannot reveal the full and exact reference of terms, "qualia" included. Understanding the reference of terms requires doing science. The exact reference of many (any?) terms doesn't exist, due to semantic vagueness. There is no exact number of hairs you need in order not to be bald. There may likewise be no exact cutoff in evolution between sentient and nonsentient organisms. But the existence of dusk and dawn doesn't stop the difference between night and day being like, well, the difference between night and day.