timtyler comments on The Human's Hidden Utility Function (Maybe) - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (87)
...but the domain of a utility function surely includes sensory inputs and remembered past experiences (the state of the agent). You are trying to assign utilities to outputs.
If you try and do that you can't even encode absolutely elementary preferences with a utility function - such as: I've just eaten a peanut butter sandwich, so I would prefer a jam one next.
If that is the only type of utility function you are considering, it is no surprise that you can't get the theory to work.