Or more costly if you factor in the aggravation of dealing with the insurance company

Presumably if that was the case, she wouldn't have bought insurance...

So, since basically everyone in the world is overconfident, you can make them better calibrated just by making them come up with an interval and then doubling it.

What I've never really got is how you become accurately calibrated at the long tails. Are there really people who can consistently give both 90% and 95% confidence intervals? To me those both just feel like "really likely", and the higher the granularity, the harder it gets - note that a 98% confidence interval should probably be twice as wide as a 95% confidence interval. Are there people who have truly internalised this?

I personally like this two player calibration game, which I was introduced to by Paul Christiano at a meetup a couple of years ago:

1. think of an unknown quantity (What year was the first woman elected to the US Congress?)
2. Player 1 comes up with a 50% confidence interval (I guess, technically, this is a credible interval...).
3. Player 2 chooses whether they want to take the "in" or the "out" side of the bet.

There's no need to choose a minimum width confidence interval (is there a technical term for that?) e.g. "before 1920" would be an acceptable confidence interval for the question given above.

The big advantage of 50% confidence intervals over 90% confidence intervals (other than that they make a nice easy structure for the game) is that you get much faster feedback. 20 trials can meaningfully tell you that your 50% confidence intervals are off in one direction or the other. 20 trials is enough to tell you if you're overconfident, but it can't tell you if you're underconfident.

The big disadvantage is that 50% confidence intervals somehow don't feel as useful as 90% confidence intervals. I'm not sure this is really true, as there's nothing special about 90% (by my reckoning 50% is about as far away from 90% as 90% is from 98%), but it feels true. Of course, it's pretty trivial to change the game so it works with intervals other than 50%, but you have to play longer, and it gets more complicated.

Not sure, as I'm not a native speaker of another language, but do most other languages use the same word for "should" in all three of those sentences? If not, it seems highly likely that it's just some sort of linguistic accident. If so, there might be something interesting worth pursuing.

The specific project I was evaluating had only gotten $800,000 out of the maximum $2m. Its strategy was to purchase the male students iPod Touches, the female students makeovers, manicures, and pedicures at a local beauty parlor, and all students were offered an additional iPod Touch or Makeover, respectively, if they passed the exam at the end of the current year.... only 25% (14/56) of the students targeted by the program had failed the reading exam in the first place.

$800,000/56 students = $14,000 per student. Those are some expensive iPod touches!

This is definitely true. General class of examples: almost any combinatorial problem ever. Concrete example: the Four Colour Theorem

Thanks, I was not trying the right combination of keywords.

(One could argue that these questions should probably be ignored and not investigated in depth - to paraphrase Teller, often magic is simply putting in more effort than any sane person would - but nevertheless, this is how things work for me.)

I can't find a source for this quote (and if it's from a longer interview, I think I'd probably like to read it), possibly because I'm not picking the right words to Google. Do you have a citation?

...that's about the last situation in which I'd expect people to rely on God

Does this cause you to doubt the veracity of the claim in the parent, or to update towards your model of what people rely on God for being wrong? I guess it should probably be both, to some extent. It's just not really clear from your post which you're doing.

I notice that http://www.miri.org is very definitely not a placeholder for a new Singularity Institute page. Have you managed to acquire it?

(miri.com seems as though it should be available, but not exactly entirely appropriate. Maybe better than nothing).