Bayesian examination
A few months ago, Olivier Bailleux, a Professor of computer science and reader of my book on Bayesianism, sent me an email. He suggested to apply some of the ideas of the book to examine students. He proposed Bayesian examination. I believe it to be a brilliant idea, which could have an important impact on how many people think. At least, I think that this is surely worth sharing here. tl;dr Bayesian examinations seem very important to deploy because they incentivize both probabilistic thinking and intellectual honesty. Yet, as argued by Julia Galef in this talk, incentives seem critical to change our thinking habits. Let's take an example Where is the International Olympic Committee? 1. Geneva 2. Lausanne 3. Zurich 4. Lugano Quite often, students are asked to select one of the four possible answers. But this is arguably pretty bad, for several reasons: - It makes impossible to distinguish a student who has a hunch from a student who really studied and knew the answer. - It gives students the habit of self-identifying with a single answer. - It normalizes deterministic question answering. - It motivates students to defend the answer they gave (which encourages the motivated reasoning fallacy...). Instead, Bayesian examination demands that students provide probabilistic answers. In other words they will have to provide percentage for each answer. In our case, a student, call her Alice, might thus answer 1. 33% 2. 33% 3. 33% 4. 1% Alice would essentially be formalizing the sentence "I really don't know but I would be very surprised if Lugano was the right answer". Another student, let's call him Bob, might answer 1. 5% 2. 40% 3. 50% 4. 5% Bob might be having in mind something like "I know that FIFA and CIO are in Zurich and Lausanne, but I don't remember which is where; though Zurich is larger so it would make sense for CIO to be in Zurich rather than Lausanne". Spoiler: the answer turns out to be Lausanne. Why naive notation is bad Now, how would such an