This is quite a bit more potent than simply assuming you're not a Boltzmann brain personally: if extrapolated to all observers, then no (or very few) Boltzmann brains need be considered. The problematic agents effectively don't exist within the parts of the model you've chosen to look at.

I'm not sure what you mean by this. I'm claiming we need not consider the possibility that we are Boltzmann brains when we are reasoning anthropically. I'm not claiming that Boltzmann brains are not observers (although they may not be), nor am I claiming that they do not exist. I also think that if a Boltzmann brain were reasoning anthropically (if it could), then it should include Boltzmann brains in its reference class. So I don't think the claims I'm making can be extrapolated to all observers. They can be extrapolated to other observers sufficiently similar to me.