In response to 2012 Survey Results
Comment author: [deleted] 29 November 2012 01:22:13PM 23 points [-]

I really have no idea what went so wrong [with the question about Bayes' birth year]

Note also that in the last two surveys the mean and median answers were approximately correct, whereas this time even the first quartile answer was too late by almost a decade. So it's not just a matter of overconfidence -- there also was a systematic error. Note that Essay Towards Solving a Problem in the Doctrine of Chances was published posthumously when Bayes would have been 62; if people estimated the year it was published and assumed that he had been approximately in his thirties (as I did), that would explain half of the systematic bias.

In response to comment by [deleted] on 2012 Survey Results
Comment author: Sam_Jaques 02 December 2012 04:01:53AM 4 points [-]

To expand on this: Confidence intervals that are accurate for multiple judgements by the same person may be accurate for the same judgement made by multiple people. Normally, we can group everyone's responses and measure how many people were actually right when they said they were 70% sure. This should average out to 70% is because the error is caused by independent variations in each person's estimate. If there's a systematic error, then even if we all accounted for the systematic error in our confidence levels, we would all still fail at the same time if there was an error.

In response to comment by gwern on How to Fix Science
Comment author: Eugine_Nier 04 March 2012 01:00:24AM 5 points [-]

I thought the usual proposals/methods involved principally reporting log odds, to avoid exactly the issue of people having varying priors and updating on trials to get varying posteriors.

This only works in extremely simple cases.

Comment author: Sam_Jaques 05 March 2012 04:00:27PM 1 point [-]

Could you give an example of an experiment that would be too complex for log odds to be useful?

Comment author: Sam_Jaques 13 October 2011 02:06:37AM 5 points [-]

Hello,

I was introduced to Less Wrong by a friend about a year ago. My first impression was of thoughts and opinions that I already had, or had half-thought, but expressed much more clearly. How could I not love it? I eventually read all of the sequences, finding novel but brilliant ideas. I now recommend them to almost everyone I meet. Coincidentally, after I'd started reading the sequences, I found HP:MOR, and had my mind blown when I found out most of them were written by the same person. Currently, I'm trying to read E.T. Jaynes', "Probability Theory: The Logic of Science", but I'm having some trouble, especially since I can't seem to solve any of the examples. If anyone has a solutions guide, or some small hints, I'd greatly appreciate it.