= 27d567bc2d48d9f40e9ccc20ea370b15)
jimrandomh comments on Bayes' Theorem Illustrated (My Way) - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (191)
If your utility function's goal is to get the most accurate hypothesis (not act on it) sure. Otherwise, why waste its time testing something that it already believes is true? If your goal is to get the highest "utility" as possible, then wasting time or resources, no matter how small, is inefficient. This means that your moving the blame off the bayesian end and to the "utility function", but its still a problem.
But you don't believe it is true; there's some probability associated with it. Consider for example, the following situation. Your friend rolls a standard pair of 6 sided dice without you seeing them. If you guess the correct total you get $1000. Now, it is clear that your best guess is to guess 7 since that is the most common outcome. So you guess 7 and 1/6th of the time you get it right.
Now, suppose you have the slightly different game where before you make your guess, you may pay your friend $1 and the friend will tell you the lowest number that appeared. You seem to think that for some reason a Bayesian wouldn't do this because they already know that 7 is most likely. But of course they would, because paying the $1 increases their expected pay-off.
In general, increasing the accuracy of your map of the universe is likely to increase your utility. Sometimes it isn't, and so we don't bother. Neither a Bayesian rationalist nor a traditional rationalist is going to try to say count all the bricks on the facade of their apartment building even though it increases the accuracy of their model. Because this isn't an interesting piece of the model that is at all likely to tell anything useful compared to other limited forms. If one was an immortal and really running low on things to do, maybe counting that would be a high priority.
Allright, consider a situation where there is a very very small probability that something will work, but it gives infinite utility (or at least extrordinarily large.) The risk for doing it is also really high, but because it is finite, the bayesian utility function will evaluate it as acceptable because of the infinite reward involved. On paper, this works out. If you do it enough times, you succeed and after you subtract the total cost from all those other times, you still have infinity. But in practice most people consider this a very bad course of action. The risk can be very high, perhaps your life, so even the traditional rationalist would avoid doing this. Do you see where the problem is? It's the fact that you only get a finite number of tries in reality, but the bayesian utility function calculates it as though you did it an infinite number of times and gives you the net utility.
Yes, you aren't the first person to make this observation. However, This isn't a problem with Bayesianism so much as with utilitarianism giving counter-intuitive results when large numbers are involved. See for example Torture v. dust specks or Pascal's Mugging. See especially Nyarlathotep's Deal which is very close to the situation you are talking about and shows that the problem seem to more reside in utilitarianism than Bayesianism. It may very well be that human preferences are just inconsistent. But this issue has very little to do with Bayesianism.
My guess at what's going on here is that you're intuitively modeling yourself as having a bounded utility function. In which case (letting N denote an upper bound on your utility), no gamble where the probability of the "good" outcome is less than -1/N times the utility of the "bad" outcome could ever be worth taking. Or, translated into plain English: there are some risks such that no reward could make them worth it -- which, you'll note, is a constraint on rewards.
I'm not sure I understand. Why put a constraint on the reward, and even if you do, why pick some arbitrary value?
That's my question for you! I was attempting to explain the intuition that generated these remarks of yours:
Because it might be false. If your utility function requires you to collect green cheese, and so you want to make a plan to go to the moon to collect the green cheese, you should know how much you'll have to spend getting to the moon, and what the moon is actually made of. And so it is written, "If you fail to achieve a correct answer, it is futile to protest that you acted with propriety."