= 27d567bc2d48d9f40e9ccc20ea370b15)
benelliott comments on St. Petersburg Mugging Implies You Have Bounded Utility - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (163)
A bigger problem is your ability to hand out arbitrarily large amounts of utility. Suppose the universe can be simulated by an N state Turing machine, this limits the number of possible states it can occupy to a finite (but probably very large) number. This in turn bounds the amount of utility you can offer me, since each state has finite utility and the maximum of a finite set of finite numbers is finite. (The reason why this doesn't automatically imply a bounded utility function is that we are uncertain of N.)
As a result of this:
P(you can offer me k utility) > 0 for any fixed k
but
P(you can offer me x utility for any x) = 0
To be honest thought, I'm not really comfortable with this, and I think Solomonoff needs to be fixed (I don't feel like I believe with certainty that the universe is computable). The real reason why you haven't seen any of my money is that I think the maths is bullshit, as I have mentioned elsewhere.
Thinking about it more, this isn't a serious problem for the dilemma. While P(you can offer me k utility) goes to zero as k goes to infinity but there's no reason to suppose it goes faster then 1/n does.
This means you can still set a similar dilemma, with a probability of you being able to offer me 2^n utility eventually becoming greater than (1/2)^n for sufficiently large n, satisfying the conditions for a St Petersburg Lottery.
That's just Pascal's mugging, though; the problem that "the utility of a Turing machine can grow much faster than its prior probability shrinks".