Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

MathijsJ comments on 0 And 1 Are Not Probabilities - Less Wrong

34 Post author: Eliezer_Yudkowsky 10 January 2008 06:58AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (128)

Sort By: Old

You are viewing a single comment's thread.

Comment author: MathijsJ 02 December 2010 02:24:59AM 4 points [-]

I'm kinda surprised that it's only been mentioned once in the comments (I only just discovered this site, really really great, by the way) and one from 2010 at that, but it seems to me that "a magical symbol to stand for "all possibilities I haven't considered" " does exist: the symbol "~" (i.e. not). Even the commenter who does mention it makes things complicated for himself: P(Q or ~Q)=1 is the simplest example of a proposition with probability 1.

The proposition is of course a tautology. I do think (but I'm not sure) that that is the only sort of statement that receives probability 1. This is in sync with Eliezer's "amount of evidence" interpretation. A bayesian update can only generate 1 if the initial proposition was of probability 1 or if the evidence was tautological (i.e. if Q then Q or, slightly less lame, if "Q or R" and "~R" then Q, where "Q or R" and "~R" are the evidence).

Skimming the comments, I saw two other proposals for "sure bets", the runner who clocked a negative time and the golf ball landing in a particular spot. That last one degenerated pretty quickly into a discussion about how many points there are in a field and on a ball. I think that's typical of such arguments: it depends on your model. Once you have your model specified the probability becomes 1 (or not) if the statement is (or isn't) tautological in the model. If the model isn't specified, then neither is the statement (what is a precise point?) and hence the probability. Ask the next man what the probability is of a runner clocking a negative time and he'll rightly respond: "Huh?" (unless he is a particularly obfuscatory know-it-all, in which case he might start blabbering about the speed of light. But then too, he makes a claim because he can ascribe meaning to the question, that is, he picks his model). So these are also tautological examples.

I think Eliezer's hold up pretty well for proposition that aren't tautological and hence empirical in nature: they require evidence and only tautological evidence will suffice for certainty.

About the problem of inserting 0's in certain standard theorems: I don't see a problem with Bayes' theorem (I'm curious about other examples). Dividing by 0 is not defined, so the probability of it raining when hell freezes over is not defined. That seems like a satisfactory arrangement.

Comment author: player_03 07 July 2011 07:52:50AM *  0 points [-]

Thanks for the analysis, MathijsJ! It made perfect sense and resolved most of my objections to the article.

I was willing to accept that we cannot reach absolute certainty by accumulating evidence, but I also came up with multiple logical statements that undeniably seemed to have probability 1. Reading your post, I realized that my examples were all tautologies, and that your suggestion to allow certainty only for tautologies resolved the discrepancy.

The Wikipedia article timtyler linked to seems to support this: "Cromwell's rule [...] states that one should avoid using prior probabilities of 0 or 1, except when applied to statements that are logically true or false." This matches your analysis - you can only be certain of tautologies.

Also, your discussion of models neatly resolves the distinction between, say, a mathematically-defined die (which can be certain to end up showing an integer between 1 and 6) and a real-world die (which cannot quite be known for sure to have exactly six stable states).

Eliezer makes his position pretty clear: "So I propose that it makes sense to say that 1 and 0 are not in the probabilities; just as negative and positive infinity, which do not obey the field axioms, are not in the real numbers."

It's true - you cannot ever reach a probability of 1 if you start at 0.5 and accumulate evidence, just as you cannot reach infinity if you start at 0 and add integer values. And the inverse is true, too - you cannot accumulate evidence against a tautology and bring its probability down to anything less than 1. But this doesn't mean a probability of 1 is an incoherent concept or anything.

Eliezer: if you're going to say that 0 and 1 are not probabilities, you need to come up with a new term for them. They haven't gone away completely just because we can't reach them.

Edit a year and a half later: I agree with the article as written, partially as a result of reading How to Convince Me That 2 + 2 = 3, and partially as a result of concluding that "tautologies that have probability 1 but no bearing on reality" is a useless concept, and that therefore, "probability 1" is a useless concept.

Comment author: fubarobfusco 29 July 2011 09:28:29AM -1 points [-]

Jaynes avoids P(A|B) for "probability of A given evidence B" and P(B) for "probability of B", preferring P(A|BX) and P(B|X) where X is one's background knowledge. This and the above leads naturally to the question of ~X: the situation in which one's "background knowledge" is false.

Assume that background knowledge X is the conjunction of a finite number of propositions. ~X is true if any of these propositions is false. If we can factor X into YZ where Y is the portion we suspect of being false — that is, if we can isolate for testing a portion of those beliefs we previously treated as "background knowledge" — then we can ask about P(A|BYZ) and P(A|B·~Y·Z).