My intuition has been pumped hard by this problem. My intuition is that it violates what we know about physics to be able to predict what each of 6 billion human beings will do confronted with the two boxes after one hour's time elapsed.
The particular physics I think is violated is quantum mechanical uncertainty. What we believe we know from quantum mechanical uncertainty is that there are a myriad of microscopic processes of which the outcome in our world cannot be predicted. We encase this result from quantum mechanics in at least two possible interpretations labeled Copenhagen and Many Worlds. But both of these interpretations have in common that for a myriad of common events starting at t1, there are multiple mutually exclusive possible outcomes possible at time t2>t1 that are, as far as either Copenhagen or MWI interpretations allow, intrinsically unpredictable at time t1. That is, at least two possible universes at time t2 are completely consistent with the single example universe at time t1: one in which one of these quantum events has turned out one way, and one in which it has turned out another way.
So now the question comes: does this have ANYTHING to do with Newcomb's problem? And it is trivial to make sure it does. During the hour I have between when the alien sets the boxes in front of me and when I must choose, I acquire a geiger counter, and I open up the stopwatch application on my iPhone. I tune the geiger counter using lead foil and possibly some medical isotopes so that it is triggering on average about once ever 60 seconds. I start the stopwatch, wait until it has run at least 15 seconds, and then stop it next time I hear a click from the geiger counter. I look at the least significant bit on the stopwatch, which is tenths-of-a-second on my iPhone. If that number is even I will pick two boxes if that number is odd, I will pick just box B.
As far as we know from Schrödinger's cat gedankedonks, the exact time of emissions of radioactive decay particles is quantumly "random." In Copenhagen, the collapse is at a random time, in many worlds, there is a different version of the universe for each possible decay time. Either way, for the Alien to have filled that box correctly he must be either 1) Able to predict the outcome of quantum phenomenon in a way that our physics currently believes is impossible 2) have flipped a coin and gotten lucky.
Now, with thousands of humans chosen to play this game, what are the chances that I am the only one chosen who includes a quantum coin toss in his choosing mechanism? Either the chances are low, in which case chances of the alien pulling off this scam are falling as 1/2^N where N is the number of quantum coin tosses among his choosers, OR the Alien is cheating.
The Alien's form of cheating might be one of many things. Perhaps he can correctly predict what SOME humans will do, and he only offers the game to those humans, in which case he will not have offered the game to me or any humans of my ilk.
My intuition has been pumped. I have been shown a gedanken problem which I think has some components equivalent to "assume a circle with four corners," or "assume 2+2=5" or some other counterfactual that is just so counter to the factuals in OUR world that pointing out this counterfactuality is the resolution to the paradox.
The things that rule out God as a good hypothesis is not his name, it is his properties. Perhaps the limited Omniscience of being able to predict reliably what any human will do in an hour when confronted with Newcomb's boxes is god-line enough to be tossed out with God from the list of good hypotheticals. It looks that way to me.
If I am right, we don't need to develop a decision theory that lets a Friendly AI self-modify to pick one box and still call the whole endeavour rational.
If you allow randomization, you have an underspecified problem again. But you can fix it easily enough by saying that Omega fills the box with the same probability that you one-box.
Here's a variant that may help your intuition. Supppose that rather than let you pick directly, Omega asks you to write a computer program that implements whatever strategy you would have used, and that program chooses one or two boxes. In that case, the prediction would be trivial, and you would certainly want to provide a program that one-boxed.
Now suppose that instead of writ...
I have not seen any place to discuss Eliezer Yudkowsky's new paper, titled Timeless Decision Theory, so I decided to create a discussion post. (Have I missed an already existing post or discussion?)