= 27d567bc2d48d9f40e9ccc20ea370b15)
Vaniver comments on Timelessness as a Conservative Extension of Causal Decision Theory - LessWrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (65)
CDT, with the right graph, one-boxes. See Spohn 2012 (hosted by lukeprog over here).
I do think this is a step towards an algorithmic way to make the right graph. But I have a problem with this part:
From where do those three logical nodes come from? And it looks to me like we're not actually using the last one- am I not also entangled with agents in universes where Omega is lying about whether or not it would have provided me with $1,000, and in those cases, shouldn't I refuse to give it $100?
That is, there seems to me to be a difference between logical uncertainty and indexical uncertainty. It makes sense to entangle across indexical uncertainty, but it doesn't make sense to entangle across logical uncertainty.
I found that handling the Counterfactual Mugging "correctly" (according to Eliezer's intuitive argument of retroactively acting on rational precommitments) requires different machinery from other problems. You're right that we don't seem to be "using" the last one, if we act under weak entanglement, and won't pay Omega $100.
The problem is that in Eliezer's original specification of the problem, he explicitly noted that, unknown to us as the player, the coin is basically weighted. Omega isn't a liar, but there aren't even any significant quantity of MWI timelines in which the coin comes up heads and Parallel!Us actually receives the money. We're trying to decide the scenario in a way that favors a version of our agent who never exists outside Omega's imagination.
I understand the notion behind this - act now according to precommitments it would have been rational to make in the past - but my own intuitions label giving Omega the money an outright loss of $100 with no real purpose, given the knowledge that the coin cannot come up heads.
This might just mean I have badly-trained intuitions! After all, if I switch mental "scenarios" to Omega being not merely a friendly superintelligence or Time Lord but an actual Trickster Matrix Lord, then all of a sudden it seems plausible that I am the prediction copy, and that "real me" might still have a chance at $1000, and I should thus pay Omega my imaginary and worthless simulated money.
The problem is, that presupposes my being willing to believe in some other universe entirely outside my own (ie: outside the simulation) in which Omega's claim to have already flipped the coin and gotten tails is simply not true. It makes Omega at least a partial liar. It confuses the hell out of me, personally.
Another version of the entanglement proposition might be able to handle this, but it sacrifices the transitivity of entanglement (to what loss, I haven't found out):
On the upside, unlike "strong entanglement", it won't trivially lose on the Prisoners' Dilemma.
Assume that the causal Bayes nets given as input to our decision algorithm contain only indexical uncertainty.
It's an interesting question where the wrong graph would ever come from in the first place, given that we can not observe causation directly. If we are to run a bunch of copies of AIXI, for example, connected to a bunch of robotic arms, and let it observe arms moving in unison, each will learn that it controls all the arms. Representation of all the arms motions as independent would require extra data.
I think Spohn also qualifies as an extension of CDT. It's been remarked before that Spohn's "intention nodes" are very similar to EY's "logical nodes" and by transitivity also CDT+E.
Disagreed. By CDT I mean calculating utilities using:
(The only modification from the wikipedia article is that I'm using Pearl's clearer notation for P(A>Oj).)
The naive CDT setup for Newcomb's problem has a causal graph which looks like B->M<-P, where B is your boxing decision, P is Omega's prediction, and M is the monetary reward you receive. This causal graph disagrees with the problem statement, as it necessarily implies that B and P are unconditionally independent, which we know is not the case from the assumption that Omega is a perfect predictor. The causal graph that agrees with the problem statement is B->P->M and B->M, in which case one-boxing is trivially the right action.
The bulk of Spohn's paper is all about how to get over the fear of backwards causation in hypothetical scenarios which explicitly allow backwards causation. You can call that an extension if you want, but it seems to me that's all in the counterfactual reasoning module, not in the decision-making module. (That is, CDT does not describe how you come up with P(Oj|do(A)), only what you do with it once you have it.)
Uh, doesn't the naive CDT setup for Newcomb's problem normally include a "my innards" node that has arrows going to both B and P? It's that that introduces the unconditional dependence between B and P. Obviously "B -> M <- P" by itself can't even express the problem because it can't represent Omega making any prediction at all.
If you decide what your innards are, and not what your action is, then this matches the problem description. If you can somehow have dishonest innards (Omega thinks I'm a one-boxer, then I can two-box), then this again violates the perfect prediction assumption.
I believe, as an empirical question, the first explicitly CDT accounts of Newcomb's problem did not use graphs, but if you convert their argument into a graph, it implicitly assumes "B -> M <- P."
Isn't the whole point of CDT that you cut any arrows from ancestor nodes with do(A) where A is your "intervention"? Obviously you can't have your innards imply your action if you explicitly violate that connection by describing your decision as an intervention.
Here is how I understood typical CDT accounts of Newcomb's problem: You have a graph given by
B <- Innards -> PandB -> M <- P.Innardsstarts with some arbitrary prior probability since you don't know your decision beforehand. You perturb the graph by deletingInnards -> Bin order to calculatep(M | do(B)), and in doing so you end up with a graph "looking like"B -> M <- P. Then the usual "dominance" arguments determine the decision regardless of the prior probability onInnards.Of course, after doing this analysis and coming up with a decision you now know (unconditionally) the value of
Band thereforeInnards, so arguably the probabilities for those should be set to 1 or 0 as appropriate in the original graph. This is generally interpreted by CDTists as a proof that this agent always two-boxes, and always gets the smaller reward.Yes. My point is that when you have a supernatural Omega, then putting any of Omega's actions in ancestor nodes of your decisions, instead of descendant nodes of your decisions, is a mistake that violates the problem description.
But if you don't delete the incoming arches on your decision nodes then it isn't CDT anymore, it's just EDT.
Which begs the question of why we should bother with CDT in the first place.
Some people claim that EDT fails at "smoking lesion" type of problems, but I think it is due to incorrect modelling or underspecification of the problem. If you use the correct model EDT produces the "right" answer.
It seems to me that EDT is superior to CDT.
(Ilya Shpitser will disagree, but I never understood his arguments)
People have known how to deal with smoking lesion (under a different name) since the 18th century (hint: the solution is not the EDT solution):
http://www.e-publications.org/ims/submission/STS/user/submissionFile/12809?confirm=bbb928f0
The trick is to construct a system that deals with things 20 times more complicated than smoking lesion. That system is recent, and you will have to read e.g. my thesis, or Jin Tian's thesis, or elsewhere to see what it is.
I have yet to see anyone advocating EDT actually handle a complicated example correctly. Or even a simple tricky example, e.g. the front door case.
You still delete incoming arcs when you make a decision. The argument is that if Omega perfectly predicts your decision, then causally his prediction must be a descendant of your decision, rather than an ancestor, because if it were an ancestor you would sever the connection that is still solid (and thus violate the problem description).
This is a shame, because he's right. Here's my brief attempt at an explanation of the difference between the two:
EDT uses the joint probability distribution. If you want to express a joint probability distribution as a graphical Bayesian network, then the direction of the arrows doesn't matter (modulo some consistency concerns). If you utilize your human intelligence, you might be able to figure out "okay, for this particular action, we condition on X but not on Y," but you do this for intuitive reasons that may be hard to formalize and which you might get wrong. When you use the joint probability distribution, you inherently assume that all correlation is causation, unless you've specifically added a node or data to block causation for any particular correlation.
CDT uses the causal network, where the direction of the arrows is informative. You can tell the difference between altering and observing something, in that observations condition things both up and down the causal graph, whereas alterations only condition things down the causal graph. You only need to use your human intelligence to build the right graph, and then the math can take over from there. For example, consider price controls: there's a difference between observing that the price of an ounce of gold is $100 and altering the price of an ounce of gold to be $100. And causal networks allow you to answer questions like "given that the price of gold is observed to be $100, what will happen when we force the price of gold to be $120?"
Now, if you look at the math, you can see a way to embed a causal network in a network without causation. So we could use more complicated networks and let conditioning on nodes do the graph severing for us. I think this is a terrible idea, both philosophically and computationally, because it entails more work and less clarity, both of which are changes in the wrong direction.
If I understand correctly, in causal networks the orientation of the arches must respect "physical causality", which I roughly understand to mean consistency with the thermodynamical arrow of time.
There is no way for your action to cause Omega's prediction in this sense, unless time travel is involved.
Yes, different Bayesian networks can represent the same probability distribution. And why would that be a problem? The probability distribution and your utility function are all that matters.
"Correlation vs causation" is an epistemic error. If you are making it then you are using the wrong probability distribution, not a "wrong" factorization of the correct probability distribution.
The problem is that this can lead to inconsistency when you have two omegas trying to predict each other.
You say "disagreed" but then end up saying what I meant in the last paragraph.
Consider that I may have read Spohn before.
I think that we're arguing about whether the label CDT refers to just the utility calculation or the combination of the utility calculation and the counterfactual module, not about any of the math. I can go into the reasons why I like to separate those two out, but I think I've already covered the basics.
I generally aim to include the audience when I write comments, which sometimes has the side effect of being insultingly basic to the person I'm responding to. Normally I'm more careful about including disclaimers to that effect, and I apologize for missing that this time.
On further thought, I would like to see someone explain exactly why I should give Omega $100. I've heard it phrased as the retroactive following-through of rational precommitments, and I've also heard it phrased as reflective self-consistency, in the sense that even if the coin was guaranteed to come up tails this time, we should pay because we like having Omega offer us bets where the expected value is good in general (as long as 1/10 coins come up heads, we break even over time, more than that and we profit). The former case, I don't know how to handle. The latter case, I think we could represent using some form of CDT+E, or CDT over a causal-and-correlative model of beliefs.
Personally, I think all of the work is being done by Omega's super-trustworthiness, and so I don't think it's a reasonable scenario to optimize for. In the real world, making a 'rational precommitment' on information you don't possess seems like the reference class of 'scams.'
(Note that I am explictly avoiding the question of what the right thing to do is; I don't think my decision theory is currently well-equipped to handle this problem, and I'm okay with that.)
Ok, I've been talking it over with Benjamin Fox some more, and I don't think Omega's trustworthiness is the issue here. The issue is basically to come up with some decision-theoretic notion of "virtue": "I should take action X because, timelessly speaking, a history in which I always respond to choice Y with action X nets me more money/utility/happiness than any other." The idea is that taking action X or not doing so in any one particular instance can change which history we're enacting, while normal decision theories reason only over the scope of a single choice-instance, with little regard for potential futures about which we don't have specific information encoded in our causal graph.
It seems to me that the impacts of being virtuous on one's potential future is enough to justify being virtuous, and one does not need to take into account the impacts of being virtuous on alternative presents one might have faced instead. (Basically, instead of trusting that Omega would have given you something in an alternate world, you are trusting that human society is perceptive enough to notice and reward enough of your virtues to justify having them.)
Yes, we agree. "I will get rewarded for this behavior in the future at a profitable rate to justify my sacrifice in the present" is a reason to "self-sacrifice" in the present. The question is how to build a decision-theory that can encode this kind of knowledge without requiring actual prescience (that is, without needing to predict the specific place and time in which the agent will be rewarded).
Even using that notion of virtue, whether giving Omega the $100 benefits you only happens if Omega is trustworthy. So Omega's trustworthiness can still be a deciding factor.
Omega's trustworthiness mostly just means we can assign a degenerate probability of 1.0 to all information we receive from Omega.