= 27d567bc2d48d9f40e9ccc20ea370b15)
Stuart_Armstrong comments on Naturalism versus unbounded (or unmaximisable) utility options - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (72)
Can a bounded agent actually do this? I'm not entirely sure.
Even so, given any distribution f, you can generate a better (dominant) distribution by taking f and adding 1 to the result. So now, as a bounded agent, you need to choose among possible distributions - it's the same problem again. What's best distribution you can specify and implement, without falling into a loop or otherwise saying yes forever?
??? Your conclusion does not follow, and is irrelevant - we care about the impact of our actions, not about hypothetical gifts that may or may not happen, and are disconnected from anything we do.
First write 1 on a piece of paper. Then start flipping coins. For every head, write a 0 after the 1. If you run out of space on the paper, ask Omega for more. When you get a tail, stop and hand the pieces of paper to Omega. This has expected value of 1/2 * 1 + 1/4 * 10 + 1/8 * 100 + ... which is infinite.
How does that relate to the claim in http://en.wikipedia.org/wiki/Turing_machine#Concurrency that "there is a bound on the size of integer that can be computed by an always-halting nondeterministic Turing machine starting on a blank tape"?
I think my procedure does not satisfy the definition of "always-halting" used in that theorem (since it doesn't halt if you keep getting heads) even though it does halt with probability 1.
That's probably the answer, as your solution seems solid to me.
That still doesn't change my main point: if we posit that certain infinite expectations are better than others (St Petersburg + $1 being better that St Petersburg), you still benefit from choosing your distribution as best you can.
Can you give a mathematical definition of how to compare two infinite/divergent expectations and conclude which one is better? If you can't, then it might be that such a notion is incoherent, and it wouldn't make sense to posit it as an assumption. (My understanding is that people have previously assumed that it's impossible to compare such expectations. See http://singularity.org/files/Convergence-EU.pdf for example.)
Not all infinite expectations can be compared (I believe) but there's lots of reasonable ways that one can say that one is better than another. I've been working on this at the FHI, but let it slide as other things became more important.
One easy comparison device: if X and Y are random variables, you can often calculate the mean of X-Y using the Cauchy principal value (http://en.wikipedia.org/wiki/Cauchy_principal_value). If this is positive, then Y is better than X.
This gives a partial ordering on the space of distributions, so one can always climb higher within this partial ordering.
Assuming you want to eventually incorporate the idea of comparing infinite/divergent expectations into decision theory, how do you propose to choose between choices that can't be compared with each other?
Random variables form a vector space, since X+Y and rX are both defined. Let V be this whole vector space, and let's define a subspace W of comparable random variables. ie if X and Y are in W, then either X is better than Y, worse, or they're equivalent. This can include many random variables with infinite or undefined means (got a bunch of ways of comparing them).
Then we simply need to select a complementary subspace W^perp in V, and claim that all random variables on it are equally worthwhile. This can be either arbitrary, or we can use other principles (there are ways of showing that even if we can't say that Z is better than X, we can still find a Y that is worse than X but incomparable to Y).
What exactly are you doing in this step? Are you claiming that there is a unique maximal set of random variables which are all comparable, and it forms a subspace? Or are you taking an arbitrary set of mutually comparable random variables, and then picking a subspace containing it?
The point might be that if all infinite expected utility outcomes are considered equally valuable, it doesn't matter which strategy you follow, so long as you reach infinite expected utility, and if that includes the strategy of doing nothing in particular, all games become irrelevant.
If you don't like comparing infinite expected outcomes (ie if you don't think that (utility) St Petersburg + $1 is better than simply St Petersburg), then just focus on the third problem, which Wei has oddly rejected.
I've often stated my worry that Omega can be used to express problems that have no real-world counterpart, thus distracting our attention away from problems that actually need to be solved. As I stated at the top of this thread, it seems to me that your third problem is such a problem.