conchis comments on Utilons vs. Hedons - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (112)
Utility means "the function f, whose expectation I am in fact maximizing". The discussion then indeed becomes whether f exists and whether it can be doubled.
That was the original point of the thread where the thought experiment was first discussed, though.
The interesting result is that if you're maximizing something you may be vulnerable to a failure mode of taking risks that can be considered excessive. This is in view of the original goals you want to achieve, to which maximizing f is a proxy - whether a designed one (in AI) or an evolved strategy (in humans).
If "we" refers to humans, then "what we value" isn't well defined.
Crap. Sorry about the delete. :(