= 27d567bc2d48d9f40e9ccc20ea370b15)
private_messaging comments on Newcomblike problems are the norm - LessWrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (108)
Yeah, the existence of classification into 'future' and 'past' and 'future' not causing 'past', and what is exactly 'future', those are - ideally - a matter of the model of physics employed. Currently known physics already doesn't quite work like this - it's not just the future that can't cause the present, but anything outside the past lightcone.
All those decision theory discussions leave me with a strong impression that 'decision theory' is something which is applied almost solely to the folk physics. As an example of a formalized decision making process, we have AIXI, which doesn't really do what philosophers say either CDT or EDT does.
Actually, I think AIXI is basically CDT-like, and I suspect that it would two-box on Newcomb's problem.
At a highly abstract level, the main difference between AIXI and a CDT agent is that AIXI has a generalized way of modeling physics (but it has a built-in assumption of forward causality), whereas the CDT agent needs you to tell it what the physics is in order to make a decision.
The optimality of the AIXI algorithm is predicated on viewing itself as a "black box" as far as its interactions with the environment are concerned, which is more or less what the CDT agent does when it makes a decision.
AIXI is a machine learning (hyper-)algorithm, hence we can't expect it to perform better than a random coin toss on a one-shot problem.
If you repeatedly pose Newcomb's problem to an AIXI agent, it will quickly learn to one-box.
Trivially, AIXI doesn't model the problem acausal structure in any way. For AIXI, this is just a matter of setting a bit and getting a reward, and AIXI will easily figuring out that setting its decision bit to "one-box" yields an higher expected reward that setting it to "two-box".
In fact, you don't even need an AIXI agent to do that: any reinforcement learning toy agent will be able to do that.
The problem you're discussing is not Newcomb's problem; it's a different problem that you've decided to apply the same name to.
It is a crucial part of the setup of Newcomb's problem that the agent is presented with significant evidence about the nature of the problem. This applies to AIXI as well; at the beginning of the problem AIXI needs to be presented with observations that give it very strong evidence about Omega and about the nature of the problem setup. From Wikipedia:
"By the time the game begins, and the player is called upon to choose which boxes to take, the prediction has already been made, and the contents of box B have already been determined. That is, box B contains either $0 or $1,000,000 before the game begins, and once the game begins even the Predictor is powerless to change the contents of the boxes. Before the game begins, the player is aware of all the rules of the game, including the two possible contents of box B, the fact that its contents are based on the Predictor's prediction, and knowledge of the Predictor's infallibility. The only information withheld from the player is what prediction the Predictor made, and thus what the contents of box B are."
It seems totally unreasonable to withhold information from AIXI that would be given to any other agent facing the Newcomb's problem scenario.
That would require the AIXI agent to have been pretrained to understand English (or some language as expressive as English) and have some experience at solving problems given a verbal explanation of the rules.
In this scenario, the AIXI internal program ensemble concentrates its probability mass on programs which associate each pair of one English specification and one action to a predicted reward. Given the English specification, AIXI computes the expected reward for each action and outputs the action that maximizes the expected reward.
Note that in principle this can implement any computable decision theory. Which one it would choose depend on the agent history and the intrinsic bias of its UTM.
It can be CDT, EDT, UDT, or, more likely, some approximation of them that worked well for the agent so far.
I don't think someone posing Newcomb's problem would be particularly interested in excuses like "but what if the agent only speaks French!?" Obviously as part of the setup of Newcomb's problem AIXI has to be provided with an epistemic background that is comparable to that of its intended target audience. This means it doesn't just have to be familiar with English, it has to be familiar with the real world, because Newcomb's problem takes place in the context of the real world (or something very much like it).
I think you're confusing two different scenarios:
- Someone training an AIXI agent to output problem solutions given problem specifications as inputs.
- Someone actually physically putting an AIXI agent into the scenario stipulated by Newcomb's problem.
The second one is Newcomb's problem; the first is the "what is the optimal strategy for Newcomb's problem?" problem.
It's the second one I'm arguing about in this thread, and it's the second one that people have in mind when they bring up Newcomb's problem.
Then AIXI ensemble will be dominated by programs which associate "real world" percepts and actions to predicted rewards.
The point is that there is no way, short of actually running the (physically impossible) experiment, that we can tell whether the behavior of this AIXI agent will be consistent with CDT, EDT, or something else entirely.
Would it be a valid instructional technique to give someone (particularly someone congenitally incapable of learning any other way) the opportunity to try out a few iterations of the 'game' Omega is offering, with clearly denominated but strategically worthless play money in place of the actual rewards?
The main issue with that is that Newcomb's problem is predicated on the assumption that you prefer getting a million dollars to getting a thousand dollars. For the play money iterations, that assumption would not hold.
The second issue with iterating Newcomb's more generally is that it gives the agent an opportunity to precommit to one-boxing. The problem is more interesting and more difficult if you face it without having had that opportunity.
Why not? People can get pretty competitive even when there's nothing really at stake, and current-iteration play money is a proxy for future-iteration real money.
I'm not sure it really makes an assumption of causality, let alone a forward one. (Apart from the most rudimentary notion that actions determine future input) . Facing an environment with two manipulators seemingly controlled by it, it wont have a hang up over assuming that it equally controls both. Indeed it has no reason to privilege one. Facing an environment with particular patterns under its control, it will assume it controls instances of said pattern. It doesn't view itself as anything at all. It has inputs and outputs, it builds a model of whats inbetween from the experience, if there are two idenical instances of it, it learns a weird model.
Edit: and what it would do in Newcombs, itll one box some and two box some and learn to one box. Or at least, the variation that values information will.
First of all, for any decision problem it's an implicit assumption that you are given sufficient information to have a very high degree of certainty about the circumstances of the problem. If presented with the appropriate evidence, AIXI should be convinced of this. Indeed, given its nature as an "optimal sequence-predictor", it should take far less evidence to convince AIXI than it would take to convince a human. You are correct that if it was presented Newcomb's problem repeatedly then in the long run it should eventually try one-boxing, but if it's highly convinced it could take a very long time before it's worth it for AIXI to try it.
Now, as for an assumption of causality, the model that AIXI has of the agent/environment interaction is based on an assumption that both of them are chronological Turing machines---see the description here. I'm reasonably sure this constitutes an assumption of forward causality.
Similarly, what AIXI would do in Newcomb's problem depends very specifically on its notion of what exactly it can control. Just as a CDT agent does, AIXI should understand that whether or not the opaque box contains a million dollars is already predetermined; in fact, given that AIXI is a universal sequence predictor it should be relatively trivial for it to work out whether the box is empty or full. Given that, AIXI should calculate that it is optimal for it to two-box, so it will two-box and get $1000. For AIXI, Newcomb's problem should essentially boil down to Agent Simulates Predictor.
Ultimately, the AIXI agent makes the same mistake that CDT makes - it fails to understand that its actions are ultimately controlled not by the agent itself, but by the output of the abstract AIXI equation, which is a mathematical construct that is accessible not just to AIXI, but the rest of the world as well. The design of the AIXI algorithm is inherently flawed because it fails to recognize this; ultimately this is the exact same error that CDT makes.
Granted, this doesn't answer the interesting question of "what does AIXI do if it predicts Newcomb's problem in advance?", because before Omega's prediction AIXI has an opportunity to causally affect that prediction.
What it doesn't do, is make an assumption that there must be physical sequence of dominoes falling on each other from one singular instance of it, to the effect.
Not at all. It can't self predict. We assume that the predictor actually runs AIXI equation.
Ultimately, it doesn't know what's in the boxes, and it doesn't assume that what's in the boxes is already well defined (there's certainly codes where it is not), and it can learn it controls contents of the box in precisely the same manner as it has to learn that it controls it's own robot arm or what ever is it that it controls. Ultimately it can do exactly same output->predictor->box contents as it does for output->motor controller->robot arm. Indeed if you don't let it observe 'its own' robot arm, and only let it observe the box, that's what it controls. It has no more understanding that this box labelled 'AIXI' is the output of what it controls, than it has about the predictor's output.
It is utterly lacking this primate confusion over something 'else' being the predictor. The predictor is representable in only 1 way, and that's an extra counter factual insertion of actions into the model.
You need to notice and justify changing the subject.
If I was to follow your line of reasoning, then CDT also one-boxes on Newcomb's problem, because CDT can also just believe that its action causes the prediction. That goes against the whole point of the Newcomb setup - the idea is that the agent is given sufficient evidence to conclude, with a high degree of confidence, that the contents of the boxes are already determined before it chooses whether to one-box or two-box.
AIXI doesn't assume that the causality is made up of a "physical sequence of dominoes falling", but that doesn't really matter. We've stated as part of the problem setup that Newcomb's problem does, in fact, work that way, and a setup where Omega changes the contents of the boxes in advance, rather than doing it after the fact via some kind of magic, is obviously far simpler, and hence far more probable given a Solomonoff prior.
As for the predictor, it doesn't need to run the full AIXI equation in order to make a good prediction. It just needs to conclude that due to the evidence AIXI will assign high probability to the obviously simpler, non-magical explanation, and hence AIXI will conclude that the contents of the box are predetermined, and hence AIXI will two-box.
There is no need for Omega to actually compute the (uncomputable) AIXI equation. It could simply take the simple chain of reasoning that I've outlined above. Moreover, it would be trivially easy for AIXI to follow Omega's chain of reasoning, and hence predict (correctly) that the box is, in fact, empty, and walk away with only $1000.
Again, folk physics. You make your action available to your world model at the time t where t is when you take that action. You propagate the difference your action makes (to avoid re-evaluating everything). So you need back in time magic.
Let's look at the equation here: http://www.hutter1.net/ai/uaibook.htm . You have a world model that starts at some arbitrary point well in the past (e.g. big bang), which proceeds from that past into the present, and which takes the list of past actions and the current potential action as an input. Action which is available to the model of the world since it's very beginning. When evaluating potential action 'take 1 box', the model has money in the first box, when evaluating potential action 'take 2 boxes', the model doesn't have money in the first box, and it doesn't do any fancy reasoning about the relation between those models and how those models can and can't differ. It just doesn't perform this time saving optimization of 'let first box content be x, if i take 2 boxes, i get x+1000 > x'.
Why do you need "back in time magic", exactly? That's a strictly more complex world model than the non-back-in-time-magic version. If Solomonoff induction results in a belief in the existence of back-in-time magic when what's happening is just perfectly normal physics, this would be a massive failure in Solomonoff induction itself. Fortunately, no such thing occurs; Solomonoff induction works just fine.
I'm arguing that, because the box already either contains the million or does not, AIXI will (given a reasonable but not particularly large amount of evidence) massively downweight models that do not correctly describe this aspect of reality. It's not doing any kind of "fancy reasoning" or "time-saving optimization", it's simply doing Solomonoff induction, and dong it correctly.
Then it can, for experiment' sake, take 2 boxes if theres something in the first box, and take 1 otherwise. The box contents are supposedly a result of computing AIXI and as such are not computable; or for a bounded approximation, not approximable. You're breaking your own hypothetical and replacing the predictor (which would have to perform hypercomputation) with something that incidentally coincides. AIXI responds appropriately.
edit: to stpop talking to one another: AIXI does not know if there's money in the first box. The TM where AIXI is 1boxing is an entireliy separate TM from one where AIXI is 2boxing. AIXI does not in any way represent any facts about the relation between those models, such as 'both have same thing in the first box'.
edit2: and , it is absoloutely correct to take 2 boxes if you don't know anything about the predictor. AIXI represents the predictor as the surviving TMs using the choice action value as omega's action to put/not put money in the box. AIXI does not preferentially self identify with the AIXI formula inside the robot that picks boxes, over AIXI formula inside 'omega'.
If you have to perform hypercomputation to even approximately guess what AIXI would do, then this conversation would seem like a waste of time/