hairyfigment comments on Pascal's Mugging, Finite or Unbounded Resources? - Less Wrong

-1 Post author: Irgy 15 October 2015 04:01AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (26)

Comments (26)

You are viewing a single comment's thread. Show more comments above.

Comment author: AlexMennen 15 October 2015 08:19:07AM 2 points [-]

I will argue that the prior for such a world should be of the order of 1/n or lower

This class of argument has been made before. The standard counterargument is that whatever argument you have for this conclusion, you cannot be 100% certain of its correctness. You should assign some nonzero probability to the hypothesis that the probability does not decrease fast enough for the correct expected utilities to be bounded. Then, taking this uncertainty into account, your expected utilities are unbounded.

The arguments relating to the bandwidth of our sensory system fail to account for (inefficient) encodings of that information which may have some configurations with arbitrarily low likelihood.

There is a positive lower bound to the probability of observing any given data (given a bound on the description length of the data), because you might just be getting random input. Given any observation that could be the result of some 1/3^^^^3 event, it could also just randomly pop into your brain for no reason with probability far greater than that. If you see a mechanism to output a random integer from 1 to 3^^^^3, and that its output was 7, you should be almost 100% confident that there was an error in your senses or your memory or your reasoning that convinced you that the mechanism works as described, etc (where "etc" means "anything other than that you observed the output of a mechanism that generates a random integer from 1 to 3^^^^3, and it was 7").

The point is that in this situation, just paying the mugger and carrying on cannot be the best course of action, because it's not the right choice if they're lying, and if they're not then it's dominated by other much larger considerations. Thus the mugging still fails, not necessarily because of the implausibility of their threat but because of the utter irrelevance of it in the face of unboundedly more important other considerations.

This totally fails to resolve the paradox. The conclusion that you should drop everything else and go all in on pursuing arbitrarily small probabilities of even more vast outcomes is, if anything, even more counter-intuitive than the conclusion that you should give the mugger $5.

Of course this doesn't really resolve the mugging itself. You could modify the scenario to replace myself having to pay with instead a small, plausible but entirely moral threat (e.g. "I'll punch that guy in the face"). I would then be motivated to make the correct moral decision regardless of bounds on my utility (though I suppose my motivation to be correct is itself bounded).

There is no reason that the "moral component" of your utility function must be linear. In fact, the boundedness of your utility function is the correct solution to Pascal's mugging.

Comment author: Irgy 15 October 2015 09:24:43AM *  -1 points [-]

This class of argument has been made before. The standard counterargument is that whatever argument you have for this conclusion, you cannot be 100% certain of its correctness. You should assign some nonzero probability to the hypothesis that the probability does not decrease fast enough for the correct expected utilities to be bounded. Then, taking this uncertainty into account, your expected utilities are unbounded.

Standard counterargument it may be but it seems pretty rubbish to me. It seems to have the form "You can't be sure you're right about X and the consequences of being wrong can be arbitrarily bad therefore do Y". This seems like a classic case of a fully general counterargument.

If I assign a non-zero probability to being wrong in my assessment of the likelihood of any possible scenario then I'm utterly unable to normalise my distribution. Thus I see this approach as an utter failure, as far as attempts to account for logical uncertainty go.

Accounting for logical uncertainty is an interesting and to my mind unsolved problem, if we ever do solve it I'll be interested to see how it impacts this scenario.

There is a positive lower bound to the probability of observing any given data...

This is exactly what I was addressing with the discussion of the dreaming/crazy theories, random sensory input is just another variant of that. And as I said there I don't see this as a problem.

The conclusion that you should drop everything else and go all in on pursuing arbitrarily small probabilities of even more vast outcomes is, if anything, even more counter-intuitive than the conclusion that you should give the mugger $5.

Certainly, and I don't honestly reach that conclusion myself. The point I make is that this collapse happens as soon as you as much as consider the possibility of unbounded resources, the mugging is an unnecessary complication. That it might still help highlight this situation is the point I'm directly addressing in the final paragraph.

There is no reason that the "moral component" of your utility function must be linear. In fact, the boundedness of your utility function is the correct solution to Pascal's mugging.

I can see compelling arguments for bounded personal utility, but I can't see compelling argument that moral catastrophes are bounded. So, as much as it would solve the mugging (and particularly an entirely morality-based version of it), I'm not convinced that it does so correctly.

Comment author: hairyfigment 16 October 2015 04:46:15PM *  0 points [-]

Either I've misunderstood the OP completely, or the prior is based on an explicit assumption of finite resources - an assumption which would ordinarily have a probability far less than 1 - (1/3^^^^3), though in everyday circumstances we can pretty much call it 'certainty'. So no, the counterargument is absolutely valid.

Also, as you should know if you read the Muggle post, Eliezer most certainly did mean Pascal's Mugging to draw attention to the failure of expected utility to converge. So you should be clearer at the start about what you think your argument does. What you have now almost seems like a quick disclaimer added when you realized the OP had failed.

(Edited to fix typo.)

Comment author: Irgy 16 October 2015 09:50:54PM -2 points [-]

Sorry, but I don't know which section of my reply this is addressing and I can't make complete sense of it.

an explicit assumption of finite resources - an assumption which would ordinarily have a probability far less than 1 - (1/3^^^^3)

The OP is broken into two main sections, one assuming finite resources and one assuming infinite.

Our universe has finite resources, why would an assumption of finite resources in an alternative universe be vanishingly unlikely? Personally I would expect finite resources with probability ~=1. I'm not including time as a "resource" here by the way, because infinite future time can be dealt with by geometric discounting and so isn't interesting.

What you have now almost seems like a quick disclaimer added when you realized the OP had failed.

It would especially help to know which quote you are referring to here.

Overall I endeavoured to show that the mugging fails in the finite case, and is nothing particularly special in the infinite case. The mugging as I see it is intended as a demonstration that large, low complexity numbers are a problem. I argue that infinite resources are a problem, but large, low complexity numbers on their own are not.

I still don't consider my arguments to have failed (though it's becoming clear that at least my presentation of them has since no-one seems to have appreciated it), I do disclaim that the mugging still raises the question of infinite resources, but reducing it to just that issue is not a failure.

I also remain firmly convinced that expected utilities (both personal and moral) can and should converge, it's just that the correct means of dealing with infinity needs to be applied, and I leave a few options open in that regard.

Powered by Reddit Powered by Reddit