First reading about Newcomb’s Problem my reaction was petty much "wow, interesting thought" and "of course I would one box, I want to win $ 1 million after all". But I had a lingering nagging feeling, that there is something wrong with the whole premise. Now, after thinking about it for a few weeks I think I have found the problem.
First of all I want to point out, that I would still one box after seeing Omega predicting 50 or 100 other people correctly, since 50 to 100 bits of evidence are enough to ovecome (nearly) any prior I have about how the universe works. Only I do not think this scenario is physically possible in our universe.
The mistake is nicely stated here:
After all, Joe is a deterministic physical system; his current state (together with the state of his future self's past light-cone) fully determines what Joe's future action will be. There is no Physically Irreducible Moment of Choice, where this same Joe, with his own exact actual past, "can" go one way or the other.
This is only true in this sense if neither MWI is true nor there are any quantum probabilistic processes, i.e., our universe allows for a true Laplace's demon (a.k.a. Omega) to exist.
If MWI is true Joe can set it up so, that "after" Omega filled the boxes and left there "will" be Everett Branches, in which Joe "will" twobox and different Everett Branches in which Joe "will" onebox.
Intuitively I think Joe could even do this with his own brain by leaving it in "undecided" mode until Omega leaves and then using an algorithm which feels "random" to decide if he oneboxes or twoboxes. But of course I would not thrust my intuition here and I do not know enough about Joe's brain to decide if this really works. So Joe would use e.g. a single photon reflected/transmitted off/through a semitransparent mirror, ensuring, that he oneboxes respectively twoboxes in say 50% of the Everett Branches.
If MWI is not true but there are quantum probabilistic process, Omega simply cannot predict the future state of the universe. So the same procedure used above would ensure that Omega cannot predict Joes decision due to true randomness.
So I would be very very VERY surprised if I saw Omega pull this trick 100 times in a row and I could somehow rule out Stage Magic (which I could not).
I am not even sure if there is any serious interpretation of quantum mechanics that allows for the strict determinism Omega would need. Would love to hear about one in the comments!
Of course from an instrumental standpoint it is always rational to firmly precommit to onebox, since the extra $1000 are not worth taking the risk. Even the model uncertainity accounts for much more than 0.001.
Perhaps Omega and the boxes are entangled with Joe in such a way that in every branch where Joe one-boxes, he finds $1,000,000, and in every branch in which he two-boxes, he finds $1,000.
I won't try to flesh this out with QM calculations, because by my standard for being able to say anything sensible about QM (being able to ace a closed-book finals exam), I can't. But from my position of complete ignorance, it seems an obvious direction to look.
I wish standards like this were (self-)applied on this site more often.