I think that's because qualia, if they exists, have no correlation with physical world, hence we cannot convey information about them with physical means. P-zombies would talk about qualia in the same terms we do, which is actually their point. The only way to 'solve' by physical means the qualia problem is therefore accept it's inexistance and trying to understand why we think there's a problem in the first place: because every other coherent solution must produce the physical effect of this solution.