magfrump comments on Why We Can't Take Expected Value Estimates Literally (Even When They're Unbiased) - Less Wrong

75 Post author: HoldenKarnofsky 18 August 2011 11:34PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (249)

Sort By: Controversial

You are viewing a single comment's thread. Show more comments above.

Comment author: magfrump 19 August 2011 01:43:40AM *  1 point [-]

I mean that using a probability distribution rather than just saying numbers clearly dispels a naive pascal's mugging. I am open to the possibility that more heavily contrived Pascal's Muggings may exist that can still exploit an unbounded utility function but I'll read that paper and see what I think after that.

Edit: From the abstract:

The agent has a utility function on outputs from the environment. We show that if this utility function is bounded below in absolute value by an unbounded computable function, then the expected utility of any input is undefined. This implies that a computable utility function will have convergent expected utilities iff that function is bounded.

What this sounds like it is saying is that literally any action under an unbounded utility function has undefined utility. In that case it just says that unbounded utility functions are useless from the perspective of decision theory. I'm not sure how it constitutes evidence that the problem of Pascal's Mugging is unresolved.