I think an analogy to math might be useful here. In one sense, you could say that mathematicians are reasoning about things that are totally divorced from the world - there's no such thing as a perfect circle or an infinitely thin line in real life. Yet once you have assumed those things as axioms, you can still do completely sane and lawful reasoning on what would follow from those axioms. Similarly, the Boltzmann brain accepts the sensory data it gets as axiomatic (as do most of us), and then proceeds to carry out lawful reasoning based on that.

I can't say I like the analogy. The point of modeling an infinitely thin line is to generalize over lines of any actual thickness. The point of modeling a perfect circle is to generalize over all the slightly ellipsoid "circles" that we want to be perfectly round. We pick out mathematical constructions and axioms based on their usefulness in some piece of reasoning we want to carry out, check them for consistency, and then proceed to use them to talk about (mostly) the real world or (for fun) fake "worlds" (which occasionally turn out to be real anyway, as with non-Euclidean geometry and general relativity).