CarlShulman comments on Pascal's Muggle: Infinitesimal Priors and Strong Evidence - Less Wrong

43 Post author: Eliezer_Yudkowsky 08 May 2013 12:43AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (404)

You are viewing a single comment's thread. Show more comments above.

Comment author: CarlShulman 06 May 2013 09:50:20PM *  4 points [-]

I hadn't considered the problem of finite axioms giving you unboundedly large likelihood ratios over your exact situation.

Really? People have been raising this (worlds with big payoffs and in which your observations are not correspondingly common) from the very beginning. E.g. in the comments of your original Pascal's Mugging post in 2007, Michael Vassar raised the point:

The guy with the button could threaten to make an extra-planar factory farm containing 3^^^^^3 pigs instead of killing 3^^^^3 humans. If utilities are additive, that would be worse.

and you replied:

Congratulations, you made my brain asplode.

Wei Dai and Rolf Nelson discussed the issue further in the comments there, and from different angles. And it is the obvious pattern-completion for "this argument gives me nigh-infinite certainty given its assumptions---now do I have nigh-infinite certainty in the assumptions?" i.e. Probing the Improbable issues. This is how I explained the unbounded payoffs issue to Steven Kaas when he asked for feedback on earlier drafts of his recent post about expected value and extreme payoffs (note how he talks about our uncertainty re anthropics and the other conditions required for Hanson's anthropic argument to go through).