this assumes Boltzmann brains that have their beliefs about reality around them in some way correlated with reality, something which does not necessarily have such a high probability

Certainly the vast majority of Boltzmann brains don't become gods, in the same way that the vast majority of virtual particles don't form brains. But it only takes one, ever.

However, it occurs to me that I haven't actually done the math, and the improbability of an AGI forming out of the ether may well exceed the space-and-time volume of the universe from Big Bang to heat death.

the vast majority of Boltzmann brains will arise after what we would normally call the heat death of the universe. They won't in general have the resources to control their future light-cone because it won't have enough usable energy.

This sounds like a plausible argument for heat death as a hard deadline on the birth of a Boltzmann god. But once one exists, any others that arise in its future light cone are rendered irrelevant.