In the paraphrased words of Eliezer, "Is there any way-the-world-could-be—any state of affairs—that corresponds to Direct Realism being true, or Indirect Realism being true?"

Shouldn't you ask those people debating indirect vs. direct realism?

Actually, Eliezer's question likely won't help, because the answers would likely turn on more structural commitments of their models of the world.

Ask the I's and D's what they'd do differently if someone convinced them to switch sides, and what they do differently now. Action is where the rubber meets the road, not truth.