wedrifid comments on Why We Can't Take Expected Value Estimates Literally (Even When They're Unbiased) - Less Wrong

75 Post author: HoldenKarnofsky 18 August 2011 11:34PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (249)

You are viewing a single comment's thread. Show more comments above.

Comment author: wedrifid 30 August 2011 01:35:35PM *  6 points [-]

Besides, intuitively if you were really offered with perfect confidence a bet with a 99.99...90% chance of winning you a dollar but if you lose all the souls in the world will burn in hell forever do you really think it would be a worthwhile bet to take?

Just how many nines do those ellipses represent? That's kind of important! I mean, if there is enough nines then the value of the dollar easily outweighs the risk and your intuition is deceiving you. Consider for example:

Given the information I currently have there is some non zero chance that I am wrong about the origin of the universe. It could be that there really is a God that will punish us with eternity in hell if we have the wrong belief about Him. A friendly superintelligence has a reasonable chance of figuring this out for us and telling us the Good News. Me having an extra dollar produces a non-zero increase in the probability that we create an AI. Multiplying non-zero chances together gives other nonzero chances. For any given probability of eternities in hell there is a number of nines such that (100 - 99.99#{n * "9"}90)% is a lower probability of eternities in hell.

Moral of this story: Be careful when throwing not-quite-infinities around to prove a point. Not-quite-infinities @#$% things up almost as much as infinities.