I am working on an article titled "You Can Gain Information Through Psychoanalysing Others", with the central thesis being with knowledge of the probability someone assigns a proposition, and their calibration, you can calculate a Bayesian probability estimate for the truthhood of that proposition.
For the article, I would need a rigorously mathematically defined system for calculating calibration given someone's past prediction history. I thought of developing one myself, but realised it would be more prudent to inquire if one has already been invented to avoid reinventing the wheel.
Thanks in advance for your cooperation. :)
#Disclaimer
I am chronically afflicted with a serious and invariably fatal epistemic disease known as narcissist bias (this is a misnomer as it refers a broad family of biases). No cure is known yet for narcissist bias, and I’m currently working on cataloguing and documenting the disease in full using myself as a test case. This disease affects how I present and articulate my points—especially in written text—such that I assign a Pr of > 0.8 that a somebody would find this post condescending, self-aggrandising, grandiose or otherwise deluded. This seems to be a problem with all my writing, and a cost of living with the condition I guess. I apologise in advance for any offence received, and inform that I do not intend to offend anyone or otherwise hurt their sensibilities.
While I don't have my notes in front of me, I do recall from the decision analysis class I recently took that log score is related to the weight one would give to one forecaster among several when combining forecasts. Unfortunately it does not appear that the professor uploaded the slides on ensemble forecasting, so I can't provide any more right now. I am emailing the professor. Thought this would help in the meantime.
I can see that I misremembered the lecture. Seems to be an application of Bayes as Lumifer suggested for the basic approach. Other more complex approaches were also discussed.