= 27d567bc2d48d9f40e9ccc20ea370b15)
Douglas_Knight comments on Open thread, Mar. 14 - Mar. 20, 2016 - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (212)
A while ago I was, for some reason, answering a few hundred questions with yes-or-no answers. I thought I would record my confidence in the answers in 5% intervals, to check my calibration. What I found was that for 60%+ confidence I am fairly well calibrated, but when I was 55% confidant I was only right 45% of the time (<50% Bayes factor 7, <55% Bayes factor >100)!
I think what happened is that sometimes I would think of a reason why the proposition X is true, and then think of some reasons why X is false, only I would now be anchored onto my original assessment that X is true. So instead of changing my mind to 'X is false' I would only decrease my confidence.
I.e. my thought processes looked like this
reason why X is true -> X is true, 60% confidence -> reasons why X is false -> X is true, 55% confidence
When it should be:
reason why X is true -> X is true, 60% confidence -> reasons why X is false -> CHANGE OPINION -> X is false, 55% confidence
Did you write the questions or were they presented to you? If they were presented to you, then you have no choice in which of the two answers is "yes" and which is "no." So it is meaningful for you distinguish between the questions for which you answered 55% and the questions for which you answered 45%. Did you find a symmetrical effect?
It was symmetric. I never answered 45% - to clarify, when I answered 55% I was right 45% of the time. And I only recorded whether I was right or wrong, not whether I was right about X being false.