except you couldn't, because there's no set of sensory inputs that corresponds to that, because they're impossible.

That does not follow. I'll admit my original example is mildly flawed, but let's tack on something (that's still impossible) to illustrate my point: invisible pink telekinetic unicorns. Still not a thing that can exist, if you define telekinesis as "action at a distance, not mediated through one of the four fundamental forces." But now, if you see an object stably floating in vacuum, and detect no gravitational or electromagnetic anomalies (and you're in an accelerated reference frame like the surface of the earth, etc etc), you can infer the presence of an invisible telekinetic something.

Or in general - an impossible object will have an impossible set of sensory inputs, but the set of corresponding sensory inputs still exists.