= 27d567bc2d48d9f40e9ccc20ea370b15)
Vaniver comments on Open Thread, November 16–30, 2012 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (213)
The problem is how classical logical statements work. The statement "If A then B" more properly translates as "~(A and ~B)".
Thus, we get valid logical statements that look bizarre to humans: "If Paris is the capital of France, then Rome is the capital of Italy" seems untrue in a causal sense (if we changed the capital of France, we would not change the capital of Italy, and vice versa) but it is true in a logical sense, because A is true, B is true, true and ~true is false, and ~false is true.
That example seems just silly, but the problem is the reverse example is disastrous. Notice that, because of the "and," if A is false then it doesn't matter what B is: false and X is false, ~false is true. If I choose the premise "Marseilles is the capital of France," then any B works. "If Marseilles is the capital of France, then I will receive infinite utility" is a true relationship under classical logic, but is clearly not a causal relationship: changing the capital will not grant me infinite utility, and as soon as the capital changes, the logical truth of the sentence will change.
If you have a reasoner that makes decisions, they need to use causal logic, not classical logic, or they'll get tripped up by the word "implication."
I get that. What I'm really wondering is how this extends to probabilistic reasoning. I can think of an obvious analog. If the algorithm assigns zero probability that it will choose $5, then when it explores the counterfactual hypothesis "I choose $5", it gets nonsense when it tries to condition on the hypothesis. That is, for all U,
is undefined. But is there an analog for this problem under uncertainty, or was my sketch correct about how that would work out?
A causal reasoner will compute about P(utility=U| do{action=$5}), which doesn't run into this trouble. This is the approach I recommend.
Probabilistic reasoning about actions that you will make is, to the best of my knowledge, not a seriously considered approach to making decisions outside of the context of mixed strategies in game theory, and even there it doesn't apply that strong, as you can see mixed strategies as putting forth a certain (but parameterized) action whose outcome is subject to uncertainty.
I don't think your sketch is correct for two reasons:
What I am saying is that I don't assume that I maximize expected utility. I take the five-and-ten problem as a proof that an agent cannot be certain that it will make the optimal choice, while it is choosing, because this leads to a contradiction. But this doesn't mean that I can't use the evidence that a choice would represent, while choosing. In this case, I can tell that U($10) > U($5) directly, so conditioning on A=$10 or A=$5 is redundant. The point is that it doesn't cause the algorithm to blow up, as long I don't think my probability of maximizing utility is 0 or 1.
It's true that A=$5 could be stronger evidence for U($5)>U($10) than A=$10 is for U($10)>U($5). But there's no particular reason to think it would be. And as long as P(U($10)>U($5)) is large enough a priori, it will swamp out the difference. As long as making a choice is evidence for that being the optimal choice, only insofar as I am confident that I make the optimal choice in general, it will provide equally strong evidence for every choice, and cancel itself out. But in cases where a particular choice is evidence of good things for other reasons (like Newcomb's problem), taking this evidence into consideration can affect my decision.
So why can't I just use the knowledge that I'll go through this line of reasoning to prove that I will choose $10 and yield a contradiction? Because I can't prove that I'll go through this line of reasoning. Simulating my decision process as part of my decision would result in infinite recursion. Now, there may be a shortcut I could use to prove what my choice will be, but the very fact that this would yield a contradiction means that no such proof exists in a consistent formal system.
(BTW, I agree that CDT is the only decision theory that works in practice, as is. I'm only addressing one issue with the various timeless decision theories)
Well, then why even update? (Or, more specifically, why assume that this is harmless normally, but an ace up your sleeve for a particular class of problems? You need to be able to reliably distinguish when this helps you and when this hurts you from the inside, which seems difficult.)
I'm not sure that I understand this; I'm under the impression that many TDT applications require that they be able to simulate themselves (and other TDT reasoners) this way.
Good questions. I don't know the answers. But like you say, UDT especially is basically defined circularly - where the agent's decision is a function of itself. Making this coherent is still an unsolved problem. So I was wondering if we could get around some of the paradoxes by giving up on certainty.