I was using the term "satisfied" to include all human preferences, including the desire to not be "straitjacketed".
If human preferences are inconsistent then humans still can't do any better than an AI for there is an AI in design space that does nothing in our world but would make similar worlds look exactly like ours.
You assume that the utility of two different worlds can not be exactly equal. edit: or maybe you don't. In any case, this AI which does absolutely nothing in our world is no more useful than AI that does nothing in all possible worlds, or just a brick.
Also, the desire for mankind (and life) not to be straitjacketed, is my view, i'm not sure it is coherently shared by mankind, and in fact i'm not even sure i like the way it is going if it is not straitjacketed in some way. edit: to clarify. I like the heuristics of maximizing the future choices for me. It i...
I laughed: SMBC comic.