All of pallas's Comments + Replies

pallas20

I agree. It seems to me that the speciality of the Necomb Problem is that actions "influence" states and that this is the reason why the dominance principle alone isn't giving the right answer. The same applies to this game. Your action (sim or not sim) determines the probability of which agent you have been all along and therefore "influences" the states of the game, whether you are X or X*. Many people dislike this use of the word "influence" but I think there are some good reasons in favour of a broader use of it (eg. quantum entanglement).

2Manfred
Actually, when working through it, it looks like pre-nap self was wrong! (Okay no distancing, I was wrong :P). I still wrote up a big comment though, you'll see what I neglected before. Short counterargument - if rather than a mild reward, the simulated copies got a horrible punishment, would that change whether you picked Sim when you might be one of those copies? Then because of how utility works the simulated copies always matter.
2Manfred
Actually, upon consideration, I will endorse answering "Sim." Will explain in a top level comment after nap. But I certainly agree that the resemblance to Newcomb's problem is why this is interesting. EDIT: Whoops, I was wrong.
pallas10

Thanks for mentioning this. I know this wasn't put very nicely.
Imagine you were a very selfish person X only caring about yourself. If I make a really good copy of X which is then placed 100 meters next to X, then this copy X only cares about the spatiotemporal dots of what we define X. Both agents, X and X, are identical if we formalize their algorithms incorporating indexical information. If we don't do that then a disparity remains, namely that X is different to X in that, intrinsically, X only cares about the set of spatiotemporal dots constituting X. ... (read more)

pallas00

It goes the other way round. An excerpt of my post (section Newcomb's Problem's problem of free will):

Perceiving time without an inherent “arrow” is not new to science and philosophy, but still, readers of this post will probably need a compelling reason why this view would be more goal-tracking. Considering the Newcomb’s Problem a reason can be given: Intuitively, the past seems much more “settled” to us than the future. But it seems to me that this notion is confounded as we often know more about the past than we know about the future. This could tempt

... (read more)
pallas10

Look, HIV patients who get HAART die more often (because people who get HAART are already very sick). We don't get to see the health status confounder because we don't get to observe everything we want. Given this, is HAART in fact killing people, or not?

It is not that clear to me what we know about HAART in this game. For instance, in case we know nothing about it and we only observe logical equivalences (in fact rather probabilistic tendencies) in the form "HAART" <--> "Patient dies (within a specified time interval)" and &qu... (read more)

2IlyaShpitser
Ok. So what is your answer to this problem: "A set of 100 HIV patients are randomized to receive HAART at time 0. Some time passes, and their vitals are measured at time 1. Based on this measurement some patients receive HAART at time 1 (some of these received HAART at time 0, and some did not). Some more time passes, and some patients die at time 2. Some of those that die at time 2 had HAART at both times, or at one time, or at no time. You have a set of records that show you, for each patient of 100, whether they got HAART at time 0 (call this variable A0), whether they got HAART at time 1 (call this variable A1), what their vitals were at time 1 (call this variable W), and whether they died or not at time 2 (call this variable Y). A new patient comes in, from the same population as the original 100. You want to determine how much HAART to give him. That is, should {A0,A1} be set to yes,yes; yes,no; no,yes; or no,no. Your utility function rewards you for keeping patients alive. What is your decision rule for prescribing HAART for this patient?" From the point of view of EDT, the set of records containing values of A0,W,A1,Y for 100 patients is all you get to see. (Someone using CDT would get a bit more information than this, but this isn't relevant for EDT). I can tell you that based on the records you see, p(Y=death | A0=yes,A1=yes) is higher than p(Y=death | A0=no,A1=no). I am also happy to answer any additional questions you may have about p(A0,W,A1,Y). This is a concrete problem with a correct answer. What is it?
pallas20

I agree that it is challenging to assign forecasting power to a study, as we're uncertain about lots of background conditions. There is forecasting power to the degree that the set A of all variables involved with previous subjects allow for predictions about the set A' of variables involved in our case. Though when we deal with Omega who is defined to make true predictions, then we need to take this forecasting power into account, no matter what the underlying mechanism is. I mean, what if Omega in Newcomb's Problem was defined to make true predictions an... (read more)

0Will_Sawin
Suppose I am deciding now whether to one-box or two-box on the problem. That's a reasonable supposition, because I am deciding now whether to one-box or two-box. There are a couple possibilities for what Omega could be doing: 1. Omega observes my brain, and predicts what I am going to do accurately. 2. Omega makes an inaccurate prediction, probabilistically independent from my behavior. 3. Omega modifies my brain to a being it knows will one-box or will two-box, then makes the corresponding prediction. If Omega uses predictive methods that aren't 100% effective, I'll treat it as combination of case 1 and 2. If Omega uses very powerful mind-influencing technology that isn't 100% effective, I'll treat it as a combination of case 2 and 3. In case 1 , I should decide now to one-box. In case 2, I should decide now to two-box. In case 3, it doesn't matter what I decide now. If Omega is 100% accurate, I know for certain I am in case 1 or case 3. So I should definitely one-box. This is true even if case 1 is vanishingly unlikely. If Omega is even 99.9% accurate, then I am in some combination of case 1, case 2, or case 3. Whether I should decide now to one-box or two-box depends on the relative probability of case 1 and case 2, ignoring case 3. So even if Omega is very accurate, ensuring that the probability of case 2 is small, if the probability of case 1 is even smaller, I should decide now to two-box. ---------------------------------------- I mean, I am describing a very specific forecasting technique that you can use to make forecasts right now. Perhaps a more precise version is, you observer children in one of two different preschools, and observe which school they are in. You observe that almost 100% of the children in one preschool end up richer than the children in the other preschool. You are then able to forecast that future children observed in preschool A will grow up to be rich, and future children observed in preschool B will grow up to be p
pallas20

If lots of subjects were using CDT or EDT, they would all be choosing ice cream independently of their soda, and we wouldn't see that correlation (except maybe by coincidence). So it doesn't have to be stated in the problem that other subjects aren't using evidential reasoning--it can be seen plainly from the axioms! To assume that they are reasoning as you are is to assume a contradiction.

If lots of subjects were using CDT or EDT, they would be choosing ice cream independently of their soda iff the soda has no influence on whether they argue according ... (read more)

0SilentCal
By 'using [CDT|EDT]', I meant 'adhering to a belief in [CDT|EDT] that predates drinking the soda.' If you're the only one (or one of the only ones) doing this, screening off would apply, right? But if others are doing this, there would be fewer correct predictions. And if you aren't doing this, you'll switch to CDT if you get CS, making your reasoning today for naught. (Doing decsion theory logically enough to overcome your subconscious desires would have the same effect as sticking to your pre-soda beliefs--either way you get an ice cream choice independent of your soda)
pallas10

Presumably, if you use E to decide in Newcomb's soda, the decisions of agents not using E are screened off, so you should only calculate the relevant probabilities using data from agents using E.

Can you show where the screening off would apply (like A screens off B from C)?

pallas10

I claim EDT is irrepairably broken on far less exotic problems than Parfit's hitchhiker. Problems like "should I give drugs to patients based on the results of this observational study?"

This seems to be a matter of screening off. Once we don't prescribe drugs because of evidential reasoning we don't learn anything new about the health of the patient. I would only not prescripe the drug if a credible instance with forecasting power (for instance Omega) shows to me that generally healthy patients (who show suspicious symptoms) go to doctors who... (read more)

2IlyaShpitser
Look, this is all too theoretical for me. Can you please go and actually read my example and tell me what your decision rule is for giving the drug? ---------------------------------------- There is more to this than d-separation. D-separation is just a visual rule for the way in which conditional independence works in certain kinds of graphical models. There is not enough conceptual weight in the d-separation idea alone to handle decision theory.
pallas10

My comment above strongly called into question whether CDT gives the right answers. Therefore I wouldn't try to reinvent CDT with a different language. For instance, in the post I suggest that we should care about "all" the outcomes, not only the one happening in the future. I've first read about this idea in Paul Almond's paper on decision theory. An excerpt that might be of interest:

Suppose the universe is deterministic, so that the state of the universe at any time completely determines its state at some later time. Suppose at the present

... (read more)
0CronoDAS
This quote seems to be endorsing the Mind Projection Fallacy; learning about the past doesn't seem to me to be the same thing as determining it...
0[anonymous]
This seems to be a matter of screening off. Once we don't prescribe drugs because of evidential reasoning we don't learn anything new about the health of the patient. I would only not prescripe the drug if a credible instance with forecasting power (for instance Omega) shows to me that generally healthy patients (who show suspicious symptoms) go to doctors who endorse evidential reasoning and unhealthy patients go to conventional causal doctors. This sounds counterintuitive, but structurally it is equal to Newcomb's Problem: The patient corresponds to the box, we know it already "has" a specific value, but we don't know it yet. Choosing only box B (or not to give the drug) would be the option that is only compatible with the more desirable past where Omega has put the million into the box (or where the patient has been healthy all along).
pallas20

In the post you can read that I am not endorsing plain EDT, as it seems to lose in problems like parfit's hitchhiker or counterfactual mugging. But in other games, for instance in Newcomblike problems, the fundamental trait of evidential reasoning seems to give the right answers (as long as one knows how to apply conditional independencies!). It sounds to me like a straw man that EDT should give foolish answers (pass on crucial issues like screening off so that for instance we should waste money to make it more likely that we are rich if we don't know our... (read more)

3IlyaShpitser
I am just saying, fix CDT, not EDT. I claim EDT is irrepairably broken on far less exotic problems than Parfit's hitchhiker. Problems like "should I give drugs to patients based on the results of this observational study?" The reason I think this is I can construct arbitrarily complicated causal graphs where getting the right answer entails having a procedure that is "causal inference"-complete, and I don't think anyone who uses EDT is anywhere near there (and if they are .. they are just reinventing CDT with a different language, which seems silly). I am not strawmanning EDT, I am happy to be proven wrong by any EDT adherent and update accordingly (hence my challenge). For example, I spent some time with Paul Christiano et al back at the workshop trying to get a satisfactory answer out of EDT, and we didn't really succeed (although to be fair, that was a tangent to the main thrust of that workshop, so we didn't really spend too much time on this).
pallas20

Assuming i), I would rather say that when Omega tells me that if I choose carrots I'll have a heart attack, then almost certainly I'm not in a freak world, but in a "normal" world where there is a causal mechanism (as common sense would call it). But the point stands that there is no necessity for a causal mechanism so that c) can be true and the game can be coherent. (Again, this point only stands as long as one's definition of causal mechanism excludes the freak case.)

0[anonymous]
Seems like there are two possible cases. Either: a) There is a causal mechanism b) None of the reasoning you might sensibly make actually works. Since the reasoning only works in the causal mechanism case, the existence of the freak world case doesn't actually make any difference, so we're back to the case where we have a causal mechanism and where RichardKennaway has explained everything far better than I have.
pallas20

I think I agree. But I would formulate it otherwise:

i) Omega's prediction are true. ii) Omega predicts that carrot-choosers have heart attacks.

c) Therefore, carrot-choosers have heart attacks.

As soon as you accept i), c) follows if we add ii). I don't know how you define "causal mechanism". But I can imagine a possible world where no biological mechanism connects carrot-choosing with heart attacks but where "accidentally" all the carrot-choosers have heart-attacks (Let's imagine running worlds on a computer countless times. One day we ... (read more)

0[anonymous]
But in that case, when Omega tells me that if I choose carrots I'll have a heart attack, then almost certainly I'm not in a freak world, and actually Omega is wrong.
pallas50

Those who pick a carrot after hearing Omega's prediction, or without hearing the prediction? Those are two very different situations, and I am not sure which one you meant.

That's a good point. I agree with you that it is crucial to keep apart those two situations. This is exactly what I was trying to address considering Newcomb's Problem and Newcomb's Soda. What do the agents (previous study-subjects) know? It seems to me that the games aren't defined precise enough.
Once we specify a game in a way that all the agents hear Omega's prediction (like in New... (read more)

pallas40

I think it is more an attempt to show that a proper use of updating results in evidential reasoners giving the right answers in Newcomblike problems. Further more, it is an attempt to show that the medical version of Solomon's Problem and Newcomb's Soda aren't defined precise enough since it is not clear what the study-subjects were aware of. Another part tries to show that people get confused when thinking about Newcomb's Problem because they use a dated perception of time as well as a problematic notion of free will.

3IlyaShpitser
(Slightly tangential): my offer to any user of EDT to correctly solve the HAART problem using EDT (here: http://lesswrong.com/lw/hwq/evidential_decision_theory_selection_bias_and/9c2t) remains open. My claim is that there is no way to correctly recover the entirety of causal inference problems using EDT without being isomorphic to CDT. ---------------------------------------- So (if you believe my claim) -- why even work with a busted decision theory?
pallas40

Thanks for the comment!

However, in the A,B-Game we assume that a specific gene makes people presented with two options choose the worse one -- please note that I have not mentioned Omega in this sentence yet! So the claim is not that Omega is able to predict something, but that the gene can determine something, even in absence of the Omega. It's no longer about Omega's superior human-predicting powers; the Omega is there merely to explain the powers of the gene.

I think there might be a misunderstanding. Although I don't believe it to be impossible tha... (read more)

0Will_Sawin
We believe in the forecasting power, but we are uncertain as to what mechanism that forecasting power is taking advantage of to predict the world. analogously, I know Omega will defeat me at Chess, but I do not know which opening move he will play. In this case, the TDT decision depends critically on which causal mechanism underlies that forecasting power. Since we do not know, we will have to apply some principles for decision under uncertainty, which will depend on the payoffs, and on other features of the situation. The EDT decision does not. My intuitions and, I believe, the intuitions of many other commenters here, are much closer to the TDT approach than the EDT approach. Thus your examples are not very helpful to us - they lump things we would rather split, because our decisions in the sort of situation you described would depend in a fine-grained way on what causal explanations we found most plausible. Suppose it is well-known that the wealthy in your country are more likely to adopt a certain distinctive manner of speaking due to the mysterious HavingRichParents gene. If you desire money, could you choose to have this gene by training yourself to speak in this way?
4Richard_Kennaway
It is required. If Omega is making true statements, they are (leaving aside those cases where someone is made aware of the prediction before choosing) true independently of Omega making them. That means that everyone with gene A makes choice A and everyone with gene B makes choice B. This strong entanglement implies the existence of some sort of causal connection, whether or not Omega exists. More generally, I think that every one of those problems would be made clear by exhibiting the causal relationships that are being presumed to hold. Here is my attempt. For the School Mark problem, the causal diagram I obtain from the description is one of these: pupil's character ----> teacher's prediction ----> final mark | | V studying ----> exam performance or pupil's character ----> teacher's prediction | | V studying ----> exam performance ----> final mark For the first of these, the teacher has waived the requirement of actually sitting the exam, and the student needn't bother. In the second, the pupil will not get the marks except by studying for and taking the exam. See also the decision problem I describe at the end of this comment. For Newcomb we have: person's qualities --> Omega's prediction --> contents of boxes | | | | V V person's decision --------------------------> payoff (ETA: the second down arrow should go from "contents of boxes" to "payoff". Apparently Markdown's code mode isn't as code-modey as I expected.) The hypotheses prevent us from performing surgery on this graph to model do(person's decision). The do() operator requires deleting all in-edges to the node operated on, making it causally independent of all of its non-descendants in the graph. The hypotheses of Newcomb stipulate that this cannot be done: every consideration you could possibly employ in making a decision is assumed
2Viliam_Bur
Those who pick a carrot after hearing Omega's prediction, or without hearing the prediction? Those are two very different situations, and I am not sure which one you meant. If some people even after hearing the Omega's prediction pick the carrot and then die of a heart attack, there must be something very special about them. They are suicidal, or strongly believe that Omega is wrong and want to prove it, or some other confusion. If people who without hearing the Omega's prediction pick the carrot and die, that does not mean they would have also picked the carrot if they were warned in advance. So saying "we should also press A here" provides no actionable advice about how people should behave, because it only works for people who don't know it.
0[anonymous]
If the scenario you describe is coherent, there has to be a causal mechanism, even if you don't know what it is. If Omega is a perfect predictor, he can't predict that carrot-choosers have heart attacks unless carrot-choosers have heart attacks.
pallas40

Correlation by itself without known connecting mechanisms or relationships does not imply causation.

The bayesian approach would suggest that we assign a causation-credence to every correlation we observe. Of course detecting confounders is very important since it provides you with updates. However, a correlation without known connecting mechanisms does imply causation. In particular it does it probabilistically. A bayesian updater would prefer talking about credences in causation which can be shifted up and downwards. It would be a (sometimes dangerous)... (read more)

pallas20

There is no contradiction to rejecting total utilitarianism and choosing torture.

For one thing, I compared choosing torture with the repugnant conclusion, not with total utilitarianism. For another thing, I didn't suspect there to be any contradiction. However, agents with intransitive dispositions are exploitable.

You can also descriptively say that, structurally, refusing total utilitarianism because of the repugnant conclusion is equal to refusing deontology because we've realise that two deontological absolutes can contradict each other. Or, mor

... (read more)
1Stuart_Armstrong
Transitive agents (eg average utilitarians) can reject the repugnant conclusion and choose torture. These things are not the same - many consistent, unexploitable agents reach different conclusions on them. Rejection of the repugnant conclusion does not come from scope neglect.
pallas110

I generally don't see why the conclusion is considered to be repugnant not only as a reaction of gut-feelings but also upon reflection, since we simply deal with another case of "dust speck vs torture", an example that illustrates how our limbic system is not adapted in a way that it could scale up emotions linearly and prevent intransitive dispositions.

We can imagine a world in which evolutionary mechanisms brought forth human brains that by some sort of limbic limitation simply cannot imagine the integer "17", whereas all the other n... (read more)

1Ghatanathoah
I have tested the theory that scope insensitivity is what makes the RC repugnant, and I have found it wanting. This is because the basic moral principles that produce the RC still produce repugnant conclusions in situations where the population is very small (only two people in the case of killing one person and replacing them with someone else). My reasoning in full is here.
2Stuart_Armstrong
There is no contradiction to rejecting total utilitarianism and choosing torture. Choosing torture and becoming a total utilitarian both involve bullet biting in ways that feel similar. But choosing torture is the natural consequences of almost all preferences, once you make them consistent and accept to make the choice. Becoming total ut is not (for instance, average utilitarians would also choose torture, but would obviously feel no compulsion to change their population ethics). You can also descriptively say that, structurally, refusing total utilitarianism because of the repugnant conclusion is equal to refusing deontology because we've realise that two deontological absolutes can contradict each other. Or, more simply, refusing X because of A is structurally the same as refusing X' because of A'. Just because one can reject total utilitarianism (or anything) for erroneous reasons, does not mean that every reason for rejecting total utilitarianism must be an error.
pallas110

Hi! I've been lurking around on the blog. I look forward to actively engage from now. Generally, I'm strongly interested in AI research, rationality in general, bayesian statistics and decision problems. I hope that I will keep on learning a lot and will also contribute useful insights for this community as it is very valuable what people here are about to do! So, see you on the "battlefield". Hi to everyone!