= 27d567bc2d48d9f40e9ccc20ea370b15)
nyan_sandwich comments on Terminal Bias - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (125)
Risk aversion as a terminal value follows pretty naturally from decreasing marginal utility. For example imagine we have a paperclip-loving agent whose utility function is equal to sqrt(x), where x is the number of paperclips in the universe. Now imagine a lottery which either creates 9 or 25 paperclips, each with 50% probability - an expected net gain of 17 paperclips. Now give the agent a choice between 16.5 paperclips or a run of this lottery. Which choice maximizes the agent's expected utility?
That's not risk aversion, it's just decreasing marginal utility. They look different to me.
And it's still not a terminal value, it would be instrumental.
They're really mathematically equivalent ways of expressing the same thing. If they look different to you that's a flaw in your intuition, you may want to correct it.
Ok, let's taboo "risk aversion", I'm talking about what a minimax algorithm does, where it comes up with possibilities, rates them by utility, and takes actions to avoid the worst outcomes. This is contrasted to a system that also computes probabilities to get expected utilities, and acts to maximize that. Sure you can make your utility function strongly concave to hack the traits of the minimax system into a utility maximizer, but saying that they are "mathematically equivalent" seems to be missing the point.
That's called "certainty effect" and no one is claiming that it's a terminal value.
Ok, thanks for the terminology help.