Some of the comments on the link by James_Miller exactly six months ago provided very specific estimates of how the events might turn out:
James_Miller:
- The odds of Russian intervening militarily = 40%.
- The odds of the Russians losing the conventional battle (perhaps because of NATO intervention) conditional on them entering = 30%.
- The odds of the Russians resorting to nuclear weapons conditional on them losing the conventional battle = 20%.
Me:
"Russians intervening militarily" could be anything from posturing to weapon shipments to a surgical strike to a Czechoslovakia-style tank-roll or Afghanistan invasion. My guess that the odds of the latter is below 5%.
A bet between James_Miller and solipsist:
I will bet you $20 U.S. (mine) vs $100 (yours) that Russian tanks will be involved in combat in the Ukraine within 60 days. So in 60 days I will pay you $20 if I lose the bet, but you pay me $100 if I win.
While it is hard to do any meaningful calibration based on a single event, there must be lessons to learn from it. Given that Russian armored columns are said to capture key Ukrainian towns today, the first part of James_Miller's prediction has come true, even if it took 3 times longer than he estimated.
Note that even the most pessimistic person in that conversation (James) was probably too optimistic. My estimate of 5% appears way too low in retrospect, and I would probably bump it to 50% for a similar event in the future.
Now, given that the first prediction came true, how would one reevaluate the odds of the two further escalations he listed? I still feel that there is no way there will be a "conventional battle" between Russia and NATO, but having just been proven wrong makes me doubt my assumptions. If anything, maybe I should give more weight to what James_Miller (or at least Dan Carlin) has to say on the issue. And if I had any skin in the game, I would probably be even more cautious.
I clearly don't. The calibration is about accurately estimating the lack of information and translating it into probabilities.
I think that's not getting the point. You thought the odds of a Russian intervention were less than 5%. Even at the time and with the information available that was probably too low. At the same time I would have rated James' predicitons as too high.
However how certain should you have been of your prediction? No the answer here is not likely to be 5%. It's at a meta level: your estimate was 5%, but with what error bars? How certain were you that you had processed and updated on all the information necessary to make such a call?
If the error bars were too big, you shouldn't have been making bets.