I agree that cosmological assumptions are needed to predict the heat death of the universe. I'd phrase it as: The expansion has to be slow enough that the effects of the expansion have to not drive the system significantly away from equilibrium. I'm not a cosmologist, so let me give a simplified example of how expansion can produce disequilibrium:

Say we had an insulated cylinder fitted with a piston filled with a gas with two chemical species in equilibrium, a high temperature form, like NO2, and a low temperature form, like N2O4. Say the gas is initially at complete thermal equilibrium (with respect to chemical degrees of freedom - ignore nuclear reactions!). Now yank the piston out faster than the NO2 can dimerize to give N2O4. The gas still cools (the kinetic energy of the molecules gets reduced by an adiabatic expansion - and this happens to the real universe too). But the gas is now left with a chemical degree of freedom in disequilibrium with the (kinetic energy) temperature.

Come to think of it, in the real universe, there is a very close analogy in the period of initial nucleosynthesis. If the expansion had been slow enough to allow full equilibrium to be maintained as the universe expanded and cooled, all that would be left would be iron-56, not hydrogen and helium. None of this violates the first or second law. Approach to thermodynamic equilibrium is inevitable for a closed system with a fixed volume. Changing the volume can drive it out of equilibrium.

The question for the distant future is what the future dynamics of the expansion are, and how they interact with the remaining degrees of freedom in the matter and energy in the universe. This is complex, and some of the parameters are not yet known.

A thermodynamic equilibrium would mean that the decreases in entropy would equal the increases. Since the decreases must be zero by the second law, the increases would also have to be zero. The laws of thermodynamics don't technically say that it has to increase, but there are things that you can't really prevent. If anything accelerates, it will emit gravitational waves, increasing entropy.