Related: Somebody flips a coin 100 times. It's come up heads each time. What are the odds it comes up heads on the 101st throw? If you're doing a probability problem, the answer is 50%. But in reality, would you bet even with 1000x1 odds in your favor on that 101st throw being a tails?
In reality, I wouldn't bet either way. It's very likely that there's a trick where the coinflip will go against whichever way I bet.
So, you no box on Newcomb's Problem? :)
The chapter on judgment under uncertainty in the (excellent) new Oxford Handbook of Cognitive Psychology has a handy little section on recent critiques of the "heuristics and biases" tradition. It also discusses problems with the somewhat-competing "fast and frugal heuristics" school of thought, but for now let me just quote the section on heuristics and biases (pp. 608-609):
My thanks to MIRI intern Stephen Barnes for transcribing this text.