If it's worth saying, but not worth its own post (even in Discussion), then it goes here.
A few weeks ago I started assessing my own calibration, using tools such as the CFAR calibration game. I got fairly good and concluded that I am relatively well calibrated.
When given a question, my instincts would immediately throw out a number. I'd unpack it and adjust it in accordance to known biases. (Avoid representativeness, start from base rates, treat the initial number as a degree of support, factor in strength of evidence, etc.)
Yesterday, an assessment of probability came up in conversation. Immediately, my instincts threw out the number "80%". My thoughts went like this:
My gut says 80%. I'm well calibrated, so 80% is probably right.
I opened my mouth to speak.
Then I shut my mouth.
I understand Löb's theorem on an intuitive level now.
Hey, thanks for mentioning this. I hadn't heard about it.
I've tried my hand at this app (50 questions or so), and it seems like the correct strategy, for me, is to go 50% for anything I have a little doubt on, and 99% for that I'm sure about. Maybe 5% of the questions fall into the 60%-90% range.
I'm still working to understand the tutorial and how to interpret my results.