TimFreeman comments on St. Petersburg Mugging Implies You Have Bounded Utility - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (163)
Predicting your preferences requires specifying both the utility function and the framework, so offering a utility function without the framework as an explanation for your preferences does not actually explain them. I actually don't know if my question was hypothetical or not. Do we have a decision procedure that gives reasonable results for an unbounded utility function?
The phrase "rule this out" seems interesting here. At any given time, you'll have a set of explanations for your behavior. That doesn't rule out coming up with better explanations later. Does the best explanation you have for your preferences that works with a known decision theory have bounded utility?
Perhaps I see what's going on here -- people who want unbounded utility are feeling loss when they imagine giving that up that unbounded goodness in order to avoid bugs like the one described in the OP. I, on the other hand, feel loss when people dither over difficult math problems when the actual issues confronting us have nothing to do with difficult math. Specifically, dealing effectively with the default future, in which one or more corporations make AI's that optimize for something having no connection to the preferences of any individual human.
Not one compatible with a Solomonoff prior. I agree that a utility function alone is not a full description of preferences.
The best explanation that I have for my preferences does not, AFAICT, work with any known decision theory. However, I know enough of what such a decision theory would look like if it were possible to say that it would not have bounded utility.
I disagree that I am doing such. Whether or not the math is relevant to the issue is a question of values, not fact. Your estimates of your values do not find the math relevant; my estimates of my values do.