Roko:

So this [that people become more cautious when the stakes are high] actually contradicts the paper that Eliezer cited, or at least seems to, yet it seems to ring true.

There is no contradiction. This paper shows that when stakes increase, people start thinking somewhat more accurately, but not drastically so -- which is exactly what I wrote above.

What these researchers did was not the sort of thing that triggers people's caution/normality/status-quo heuristics that I had in mind. They put people in a situation where they stood only to gain free money, and were forced to choose between several options, each with a guaranteed non-negative outcome. A study that would actually test my claims would observe people in a situation where they could lose a significant amount of their own money by accepting a bad deal based on biased reasoning.

Of course, this actually happens sometimes, for example with people who ruin themselves by compulsive gambling. But these are rare exceptions, not instances of all-pervasive systematic biases.

Feel free to PM me and claim your prize.

I don't have a Paypal account, but you can buy me a beer next time I'm over in the U.K. :-)