You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

hairyfigment comments on Pascal's Mugging, Finite or Unbounded Resources? - Less Wrong Discussion

-1 Post author: Irgy 15 October 2015 04:01AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (26)

Comments (26)

You are viewing a single comment's thread. Show more comments above.

Comment author: Irgy 15 October 2015 09:24:43AM *  -1 points [-]

This class of argument has been made before. The standard counterargument is that whatever argument you have for this conclusion, you cannot be 100% certain of its correctness. You should assign some nonzero probability to the hypothesis that the probability does not decrease fast enough for the correct expected utilities to be bounded. Then, taking this uncertainty into account, your expected utilities are unbounded.

Standard counterargument it may be but it seems pretty rubbish to me. It seems to have the form "You can't be sure you're right about X and the consequences of being wrong can be arbitrarily bad therefore do Y". This seems like a classic case of a fully general counterargument.

If I assign a non-zero probability to being wrong in my assessment of the likelihood of any possible scenario then I'm utterly unable to normalise my distribution. Thus I see this approach as an utter failure, as far as attempts to account for logical uncertainty go.

Accounting for logical uncertainty is an interesting and to my mind unsolved problem, if we ever do solve it I'll be interested to see how it impacts this scenario.

There is a positive lower bound to the probability of observing any given data...

This is exactly what I was addressing with the discussion of the dreaming/crazy theories, random sensory input is just another variant of that. And as I said there I don't see this as a problem.

The conclusion that you should drop everything else and go all in on pursuing arbitrarily small probabilities of even more vast outcomes is, if anything, even more counter-intuitive than the conclusion that you should give the mugger $5.

Certainly, and I don't honestly reach that conclusion myself. The point I make is that this collapse happens as soon as you as much as consider the possibility of unbounded resources, the mugging is an unnecessary complication. That it might still help highlight this situation is the point I'm directly addressing in the final paragraph.

There is no reason that the "moral component" of your utility function must be linear. In fact, the boundedness of your utility function is the correct solution to Pascal's mugging.

I can see compelling arguments for bounded personal utility, but I can't see compelling argument that moral catastrophes are bounded. So, as much as it would solve the mugging (and particularly an entirely morality-based version of it), I'm not convinced that it does so correctly.

Comment author: hairyfigment 16 October 2015 04:46:15PM *  0 points [-]

Either I've misunderstood the OP completely, or the prior is based on an explicit assumption of finite resources - an assumption which would ordinarily have a probability far less than 1 - (1/3^^^^3), though in everyday circumstances we can pretty much call it 'certainty'. So no, the counterargument is absolutely valid.

Also, as you should know if you read the Muggle post, Eliezer most certainly did mean Pascal's Mugging to draw attention to the failure of expected utility to converge. So you should be clearer at the start about what you think your argument does. What you have now almost seems like a quick disclaimer added when you realized the OP had failed.

(Edited to fix typo.)

Comment author: Irgy 16 October 2015 09:50:54PM -2 points [-]

Sorry, but I don't know which section of my reply this is addressing and I can't make complete sense of it.

an explicit assumption of finite resources - an assumption which would ordinarily have a probability far less than 1 - (1/3^^^^3)

The OP is broken into two main sections, one assuming finite resources and one assuming infinite.

Our universe has finite resources, why would an assumption of finite resources in an alternative universe be vanishingly unlikely? Personally I would expect finite resources with probability ~=1. I'm not including time as a "resource" here by the way, because infinite future time can be dealt with by geometric discounting and so isn't interesting.

What you have now almost seems like a quick disclaimer added when you realized the OP had failed.

It would especially help to know which quote you are referring to here.

Overall I endeavoured to show that the mugging fails in the finite case, and is nothing particularly special in the infinite case. The mugging as I see it is intended as a demonstration that large, low complexity numbers are a problem. I argue that infinite resources are a problem, but large, low complexity numbers on their own are not.

I still don't consider my arguments to have failed (though it's becoming clear that at least my presentation of them has since no-one seems to have appreciated it), I do disclaim that the mugging still raises the question of infinite resources, but reducing it to just that issue is not a failure.

I also remain firmly convinced that expected utilities (both personal and moral) can and should converge, it's just that the correct means of dealing with infinity needs to be applied, and I leave a few options open in that regard.

Powered by Reddit Powered by Reddit