But isn't the point to find something where it's sort of unexpected to express the belief as a probability? I may have misunderstood Yvain's intention, but I don't think he was looking for examples where everybody already expresses them in probabilities.
I thought he was trying to figure out how to explain the important epistemological point that an empirical belief must be a probability (range somewhere 0-100%, usually not at either extreme), because a lot of people seem to treat a large category of empirical questions as "you think for a while, and then make up your mind".
I'm not sure whether I'm being clear enough here, but perhaps I could at least make my position seem a little bit more likely by pointing out that it seems highly unlikely that Yvain would have made a whole discussion topic about this if answering it was as easy as simply pointing to a gigantic, well-known category of beliefs that are always expressed in probabilities. It would be like asking for an example math problem that uses exponents.
Actually I might have just thought of a much better way to put it. I think his intention was to find a good example to include in an attempt to explain the classic LW assertion that probability is not in the territory. Everybody knows that certain things are usually expressed in probabilities (like whether the coin will be heads or tails), but most people don't realize that every thought process is a probability, and you can assign a probability to every belief depending on how likely you think it is that you thought through it properly.
In fact, maybe that invalidates me qualifying my writing earlier with just empirical beliefs. Isn't it the view on here that every belief is a probability, because a probability can also be you gauging how likely it is for your thought process to be sound, rather than something in the territory? Can even a non-empirical question have a probability, just not one that one must come to via frequentist methods (testing a bunch of times and seeing the ratio)?
I don't know any probability theory, so maybe I'm way off here. This post turned from a random observation to a winding attempt to grapple with some (perhaps easy) problems. Anybody who has any thoughts on the matter, it would be appreciated.
Nitpick: I think that most empirical beliefs are very close to the 0 or 100% ends, so much so that we don't even feel any uncertainty.
That said, you make an excellent point and I think you're right about Yvain's goal here.
When I explain to people how beliefs should be expressed in probabilities, I would like to use an example like "Consider X. Lots of intelligent people believe X, but lots of equally intelligent people believe not-X. It would be ridiculous to say you are 100% sure either way, so even if you have a strong opinion about X, you should express your belief as a probability."
Trouble is, I'm having a hard time thinking of an example to plug into X. For an example to work, it would need the following properties:
Factual question. So no value-laden questions like "Is abortion morally acceptable?" or counterfactual questions like "Would the US have won a war with the Soviet Union in 1960?"
Popular and important question. The average person should be aware it's an issue and care about the answer. So no "My aunt's middle name is Gladys" or "P = NP."
High uncertainty. Reasonable people should be reluctant to give a probability >90% or <10%
No opportunity to gain status by signaling overwhelming support for one side. So cryonics is out, because it's too easy to say "That's stupid, I'm 100% sure cryonics won't work and no intelligent person could believe it." I'm assuming in any debate where you can gain free status by assigning crazy high probabilities to the "responsible" position, people will do just that - so no psi effects, Kennedy assassination, or anything in that cluster. I need a question with no well-accepted "responsible" position.
Minimal mindkilling effect. My previous go-to example has been global warming, but I keep encountering people who say that global warming 100% for sure exists and the only people who could possibly doubt it are oil company shills. Or if I were to try the existence of God, I predict half the population would say it's 100% certain God exists, and the other half would say the opposite.
So what are the important questions that average (or somewhat-above-average) people will likely agree are complicated open questions where both sides have good points? And if there aren't many such questions, what does that say about us?