Yes, I agree that the politicisation is the central issue. But this is exactly why I wrote the first part - I feel that this section is true despite it (I didn't claim that most people agree with the solution, only that the elites, experts, and the reader's social bubble does!).
So one question I'm trying to understand is: since politicisation happened to climate change, why do we think that it won't happen to AI governance? I.e. the point is that pursuing goals by political means might just usually end up like that, because of the basic structure of the political discourse (you get points for opposing the other side, etc).
Hm, so one comment is that the proof in the post was not meant to convey the intuition for the existence of the concrete probability distribution - the measurability of the POWER inequality is a necessary first step, but not really technically related to the (potential) rest of the proof (although I had initially hoped that lifting some distribution on rewards by the Giry monad might produce something interesting).
As for why the additional structure might be helpful: the issue with there being no Lebesgue-like uniform measure is that in the infinite-dimens...
Thanks for the comment! Note that we use state-action visitation distribution, so we consider trajectories that contain actions as well. This makes it possible to invert η (as long as all states are visited). Using only states trajectories, it would indeed be impossible to recover the policy.