Comment on Open thread, Mar. 2 - Mar. 8, 2015

Mon, 16 Mar 2015 11:50:15 GMT

What does it mean for a decision algorithm to fail?

When it makes decisions that are undesirable. There is no point deciding to run a decision algorithm which is perfectly consistent but results in outcomes you don't want.

In the case of the Omega's-life-tickets scenario, one could argue it fails in an objective sense since it will never stop buying tickets until it dies. But that wasn't even the point I was trying to make.

And an unbelievable premise leads to an unbelievable conclusion.

I don't know if there is a name for this fallacy but there should be. Where someone objects to the premises of a hypothetical situation intended just to demonstrate a point. E.g. people who refuse to answer the trolley dilemma and instead say "but that will probably never happen!" It's very frustrating.

EU is not "required" to trade away huge amounts of outcome-space for really good but improbable outcomes. EU applies preference models to novel situations, not to produce preferences but to preserve them. If you gave EU a preference model that matched your preferences, it will preserve the match and give you actions that best satisfy your preferences in underneath the uncertainty model of the universe you gave it.

This is very subtle circular reasoning. If you assume your goal is to maximize the expected value some utility function, then maximizing expected utility can do that if you specify the right utility function.

What I've been saying from the very beginning is that there isn't any reason to believe there is any utility function that will produce desirable outcomes if fed to an expected utility maximizer.

I think a decision algorithm fails if it makes you predictably worse off than an alternative algorithm

Even if you are an EU maximizer, EU will make you "predictably" worse off, as in the majority of cases you will be worse off. A true EU maximizer doesn't care so long as the utility of the very low probability outcomes is high enough.

Discuss