@EvanWard97
Longtermist, singularitarian. Differential progress, buying time for AI alignment; epistemic infrastructure, prior elicitation, web dev.
To maximize utility when you can play any N number of games, I believe you just need to calculate the EV (not EU) through playing every possible strategy. Then, you pass all those values through your U function and go with the strategy associated with the highest utility.
<Tried to retract this comment since I no longer agree with it, but it doesn't seem to be working>
It's great to see other people thinking and working on these ideas of efficiently eliciting preferences and very 'subjective' data, and building your own long-term decision support system! I've been pretty frustrated by the seeming lack of tooling for this. Inspired partially by Gwern's Resorter as well, I've started experimenting with my own version, except my goal is to end up with random variables for cardinal utilities (at least across various metrics), and I'm having the inputs for comparisons be quickly-drawn probability distributions.