(Thanks for discussing!)

I will address your last paragraph first. The only significant difference between my original example and the proper Newcomb's paradox is that, in Newcomb's paradox, Omega is made a predictor by fiat and without explanation. This allows perfect prediction and choice to sneak into the same paragraph without obvious contradiction. It seems, if I try to make the mode of prediction transparent, you protest there is no choice being made.

From Omega's point of view, its Newcomb subjects are not making choices in any substantial sense, they are just predictably acting out their own personality. That is what allows Omega its predictive power. Choice is not something inherent to a system, but a feature of an outsider's model of a system, in much the same sense as random is not something inherent to a Eeny, meeny, miny, moe however much it might seem that way to children.

As for the rest of our disagreement, I am not sure why you insist that CDT must work with a misleading model. The standard formulation of Newcomb's paradox is inconsistent or underspecified. Here are some messy explanations for why, in list form:

• Omega predicts accurately, then you get to choose is a false model, because Omega has predicted you will two-box, then you get to choose does not actually let you choose; one-boxing is an illegal choice, and two-boxing the only legal choice (In Soviet Russia joke goes here)
• You get to choose, then Omega retroactively fixes the contents of the boxes is fine and CDT solves it by one-boxing
• Omega tries to predict but is just blindly guessing, then you really get to choose is fine and CDT solves it by two-boxing
• You know that Omega has perfect predictive power and are free to be committed to either one- or two-boxing as you prefer is nowhere near similar to the original Newcomb's formulation, but is obviously solved by one-boxing
• You are not sure about Omega's predictive power and are torn between trying to 'game' it and cooperating with it is not Newcomb's problem
• Your choice has to be determined by a deterministic algorithm, but you are not allowed to know this when designing the algorithm, so you must instead work in ignorance and design it by a false dominance principle is just cheating

Easy explanation for the Ellsberg Paradox: We humans treat the urn as if it was subjected to two kinds of uncertainties.

• The first kind is which ball I will actually draw. It feels "truly random".
• The second kind is how many red (and blue) balls there actually are. This one is not truly random.

Somehow, we prefer to chose the "truly random" option. I think I can sense why: when it's "truly random", I know no potentially hostile agent messed up with me. I mean, I could chose "red" in situation A, but then the organizers could have put 60 blue balls just to mess with me!

Put it simply, choosing "red" opens me up for external sentient influence, and therefore risk being outsmarted. This particular risk aversion sounds like a pretty sound heuristic.

I'm finding the "counterfactual mugging" challenging. At this point, the rules of the game seem to be "design a thoughtless, inert, unthinking algorithm, such as CDT or EDT or BT or TDT, which will always give the winning answer." Fine. But for the entire range of Newcomb's problems, we are pitting this dumb-as-a-rock algo against a super-intelligence. By the time we get to the counterfactual mugging, we seem to have a scenario where omega is saying "I will reward you only if you are a trusting rube who can be fleeced." Now, if you are a trusting rube who can be fleeced, then you can be pumped, a la the pumping examples in previous sections: how many times will omega ask you for $100 before you wisen up and realize that you are being extorted?

This shift of focus to pumping also shows up in the Prisoner's dilemma, specifically, the recent results from Freeman Dyson & William Press. They point out that an intelligent agent can extort any evolutionary algorithm. Basically, if you know the zero-determinant strategy, and your opponent doesn't, than you can mug the opponent (repeatedly). I think the same applies for the counterfactual mugging: omega has a "theory of mind", while the idiot decision algo fails to have one. If your decision algo tries to learn from history (i.e. from repeated muggings), using basic evolutionary algo's, then it will continue to be mugged (forever): it can't win.

To borrow Press & Dyson's vocabulary: if you want to have an algorithmic decision theory that can win in the presence of (super-)intelligences, then you must endow that algorithm with a "theory of mind": you're algorithm has got to start modelling omega, to determine what its actions will be.

The conclusion to section "11.1.3. Medical Newcomb problems" begs a question which remains unanswered: -- "So just as CDT “loses” on Newcomb’s problem, EDT will "lose” on Medical Newcomb problems (if the tickle defense fails) or will join CDT and "lose" on Newcomb’s Problem itself (if the tickle defense succeeds)."

If I was designing a self-driving car and had to provide an algorithm for what to do during an emergency, I may choose to hard-code CDT or EDT into the system, as seems appropriate. However, as an intelligent being, not a self-driving car, I am not bound to always use EDT or always use CDT: I have the option to carefully analyse the system, and, upon discovering its acausal nature (as the medical researchers do in the second study) then I should choose to use CDT; else I should use EDT.

So the real question is: "Under what circumstances should I use EDT, and when should I use CDT"? Section 11.1.3 suggests a partial answer: when the evidence shows that the system really is acausal, and maybe use EDT the rest of the time.

Presentation of Newcomb's problem in section 11.1.1. seems faulty. What if the human flips a coin to determine whether to one-box or two-box? (or any suitable source of entropy that is beyond the predictive powers of the super-intelligence.) What happens then?

This point is danced around in the next section, but never stated outright: EDT provides exactly the right answer if humans are fully deterministic and predictable by the superintelligence. CDT gives the right answer if the human employs an unpredictable entropy source in their decision-making. It is the entropy source that makes the decision acausal from the acts of the super-intelligence.

There is one rather annoying subtext that recurs throughout the FAQ: the very casual and carefree use of the words "rational" and "irrational", with the rather flawed idea that following some axiomatic system (e.g. VNM) and Bayes is "rational" and not doing so is "irrational". I think this is a dis-service, and, what's more, fails to look into the effects of intelligence, experience, training and emotion. The Allias paradox scratches the surface, as do various psych experiments. But ...

The real question is "why does this or that model differ from human nature?" : this question seems to never be asked overtly, but it does seem to get an implicit answer: because humans are irrational. I don't like that answer: I doubt that they are irrational per-se, rather, they are reacting to certain learned truths about the environment, and incorporating that into judgments,

So, for example: every day, we are bombarded with advertisers forcing us to make judgments: "if you buy our product, you will benefit in this way." which is a decision-theoretic decision based on incomplete information. I usually make a different choice: "you can choose to pay attention to this ad, or to ignore it: if you pay attention to this ad, you trade away some of you attention span, in return for something that might be good; but if you ignore it, you sacrifice nothing, but win nothing." I make this last choice hundreds of times a day. Maybe thousands. I am one big optimized mean green decision machine.

The advertizers have trained us in certain ways: in particular, they have trained us to disbelieve their propositions: they have a bad habit of lying, of over-selling and under-delivering. So when I see offers like "a jar contains red blue and yellow balls..." my knee-jerk reaction is "bullshit, I know that you guys are probably trying to scam me, and I'd be an idiot for picking blue instead of yellow, because I know that most typical salespeople have already removed all the blue marbles from the jar. Only a gullible fool would believe otherwise, so cut it out with that Bayesian prior snow-job. We're not country bumpkins, you know." 

The above argument, even if made ex-post-facto, is an example of the kind of thinking that humans engage in regularly. Humans make thousands of decisions a day (Should I watch TV now? Should I go to the bathroom? Should I read this? What should I type as the next word of this sentence?) and it seems awfully naive to claim that if any of these decisions don't follow VNM+Bayes, they are "irrational". I think its discounting intelligence far more than it should.

There are numerous typos throughout the thing. Someone needs to re-read it. The math in "8.6.3. The Allais paradox" is all wrong, option 2A is not actually 34% of 1A and 66% of nothing, etc.