Yes, your second paragraph gets at what I was thinking (and you are right that it is not exactly the Boltzmann Brain problem). But I don't think it is the same as the general problem of induction, either.

On your model, if I understand correctly, there are microscopic, time symmetric laws that hold everywhere. (That they hold everywhere and not just on our experience we take for granted--we are not allowing Humean worries about induction while doing physics, and that's fine.) But on top of that, there is a macroscopic law that we observe, the Second Law, and you are proposing (I think--maybe I misunderstand you) that its explanation lies in that we are agents and observers, and that the immediate environment of a system that is an agent and observer must exhibit this kind of time asymmetry. But then, we should not expect this macroscopic regularity to hold beyond our immediate environment. I think this is ordinary scientific reasoning, not Humean skepticism.