= 27d567bc2d48d9f40e9ccc20ea370b15)
endoself comments on Testing lords over foolish lords: gaming Pascal's mugging - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (18)
There's still the failure of convergence. If the theory that made you think that it would be a good idea to accept Pascal's mugging tells you to sum an infinite series, and that infinite series diverges, then the theory is wrong.
The convergence can be solved using the arguments I presented in: http://lesswrong.com/lw/giu/naturalism_versus_unbounded_or_unmaximisable/
Essentially, take advantage of the fact that we are finite state probabilistic machines (or analogous to that), and therefore there is a maximum to the number of choices we can expect to make. So our option set is actually finite (though brutally large).
I'm referring to an infinity of possible outcomes, not an infinity of possible choices. This problem still applies if the agent must pick from a finite list of actions.
Specifically, I'm referring to the problem discussed in this paper, which is mostly the same problem as Pascal's mugging.
Interesting problem, thanks! I personally felt that there could be a good case made for insisting your utility be bounded, and that paper's an argument in that direction.
Pascal's mugging is less of a problem if your utility function is bounded, and it completely goes away if the bound is reasonably low, since then there just isn't any amount of utility that would outweight the improbability of the mugger being truthful.
Weren't you working on ways to compare infinite/divergent expectations? I'm confused that you're now writing as if the problem is new to you...
I was working on a way to do that - but I'm also aware that not all divergent expectations can be compared, and so that there might be a case to avoid using unbounded utilities.