= 27d567bc2d48d9f40e9ccc20ea370b15)
Juno_Watt comments on Pascal's Muggle: Infinitesimal Priors and Strong Evidence - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (404)
There is no real way of doing that without changing your probability function or your utility function. However you can't change those. The real problem is with the expected utility function and I don't see any way of fixing it, though perhaps I missed something.
Any agent subject to Pascal's Mugging would fall pray to this problem first, and it would be far worse. While the mugger is giving his scenario, the agent could imagine an even more unlikely scenario. Say one where the mugger actually gives him 3^^^^^^3 units of utility if he does some arbitrary task, instead of 3^^^3. This possibility immediately gets so much utility that it far outweighs anything the mugger has to say after that. Then the agent may imagine an even more unlikely scenario where it gets 3^^^^^^^^^^3 units of utility, and so on.
I don't really know what an agent would do if the expected utility of any action approached infinity. Perhaps it would generally work out as some things would approach infinity faster than others. I admit I didn't consider that. But I don't know if that would necessarily be the case. Even if it is it seems "wrong" for expected utilities of everything to be infinite and only tiny probabilities to matter for anything. And if so then it would work out for the pascal's mugging scenario too I think.
There is no evidence for the actual existence of neatly walled-of and unupdateable utility functions or probability functions, any more than there is for a luz'.
Utility and probability functions are not perfect or neatly walled off. But that doesn't mean you should change them to fix a problem with your expected utility function. The goal of a probability function is to represent the actual probability of an event happening as closely as possible. And the goal of a utility function is to represent what you states you would prefer the universe to be in. This also shouldn't change unless you've actually changed your preferences.
There's plenty of evidence of people changing their preferences over significant periods of time: it would be weird not to. And I am well aware that the theory of stable utility functions is standardly patched up with a further theory of terminal values, for which there is also no direct evidence.
Of course people can change their preferences. But if your preferences are not consistent you will likely end up in situations that are less preferable than if you had the same preferences the entire time. It also makes you a potential money pump.
What? Terminal values are not a patch for utility functions. It's basically another word that means the same thing, what state you would prefer the world to end up in. And how can there be evidence for a decision theory?
Well, I've certainly seen discussions here in which the observed inconsistency among our professed values is treated as a non-problem on the grounds that those are mere instrumental values, and our terminal values are presumed to be more consistent than that.
Insofar as stable utility functions depend on consistent values, it's not unreasonable to describe such discussions as positing consistent terminal values in order to support a belief in stable utility functions.
Well, how is this different from changing our preferences to utility functions to fix problems with our naive preferences?
I don't know what you mean. All I'm saying is that you shouldn't change your preferences because of a problem with your expected utility function. Your preferences are just what you want. Utility functions are just a mathematical way of expressing that.
Human preferences don't naturally satisfy the VNM axioms, thus by expressing them as a utility function you've already changed them.
I don't see why our preferences can't be expressed by a utility function even as they are. The only reason it wouldn't work out is if there were circular preferences, and I don't think most peoples preferences would work out to be truly circular if they were to think about the specific occurrence and decide what they really preferred.
Though mapping out which outcomes are more preferred than others is not enough to assign them an actual utility, you'd somehow have to guess how much more preferable one outcome is to another quantitatively.But even then I think most people could if they thought about it enough. The problem is that our utility functions are complex and we don't really know what they are, not that they don't exist.
Or they might violate the independence axiom, but in any case what do you mean by " think about the specific occurrence and decide what they really preferred", since the result of such thinking is likely to depend on the exact order they thought about things in.