# Kindly comments on Irrationality Game II - Less Wrong Discussion

13 03 July 2012 06:50PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Sort By: Best

You are viewing a single comment's thread.

Comment author: 03 July 2012 10:51:20PM 22 points [-]

Irrationality game

0 and 1 are probabilities. (100%)

Comment author: 03 July 2012 11:08:21PM 11 points [-]

Nothing we can say will change your mind, unless already don't believe this.

Comment author: 04 July 2012 11:49:55PM 5 points [-]

Downvoted for agreement. Trivially, P(A|A)=1 and P(A|~A)=0.

Comment author: 04 July 2012 12:05:21AM 11 points [-]

Upvoted not for the claim, but the ridiculously high confidence in that claim.

Comment author: 04 July 2012 12:14:58AM 3 points [-]

Are you saying that you probably agree that 0 and 1 are probabilities, but my claim is not one of the things you would assign a probability of 1 to?

Comment author: 04 July 2012 12:29:06AM 2 points [-]

I believe 0 and 1 are probabilities, but there is no way to obtain that degree of certainty. (unless you have an incredibly clever method you aren't sharing, which is mean)

An analogy would be that I believe that 3^^^3 is a number, even though I don't think I will ever have that many dollars. Similarly, I believe that 0 and 1 are probabilities, but I wouldn't grant any particular belief a probability of 0 or 1.

Comment author: 04 July 2012 12:36:49AM 0 points [-]

My intention was to make a stronger claim than the one you agree with, but fortunately my degree of confidence takes care of that for me.

Comment author: 05 July 2012 11:35:55PM 1 point [-]

I'd like to point out that anyone who does not share the (claimed) Infinite Certainty should be upvoting, as this confidence level is infinitely higher than any other possible confidence level. (It's kind of like, if you agree that dividing by zero is merely an error, then any claim to infinite certainty is also an error, almost exactly the same error in fact.)

Comment author: 04 July 2012 01:55:33AM 1 point [-]

Only Sith deal in absolutes!

I am very happy that the parent is currently at 0 karma.

Comment author: 05 July 2012 01:35:04AM 1 point [-]

So you are more confident in math than in hallucinating this entire interaction with an internet forum?

Comment author: 05 July 2012 01:57:59AM 1 point [-]

I'm not quite sure how to parse that, but I'll do my best. I am more confident in math than I am in my belief that arbitrary parts of my life are not hallucinations.

Comment author: 05 July 2012 02:01:28AM 0 points [-]

Damn... You're good. Anyway, 1 and 0 aren't probabilities because Bayes Theorem break down there (in the log-odds/information base where Bayes Theorem is simple addition, they are positive and negative infinity). You can however meaningfully construct limits of probabilities. I prefer the notation (1 -) epsilon.

Comment author: 05 July 2012 02:20:24AM *  2 points [-]

Log-odds aren't what probability is, they're a way to think about probability. They happen not to work so well when the probabilities are 0 and 1; they also fail rather dramatically for probability density functions. That doesn't mean they don't have their uses.

Similarly, Bayes's Theorem breaks down because the proof of it assumes a nonzero probability. This isn't fixed by defining away 0 and 1, because it can still return those as output, and then you end up looking silly. In many cases, not being able to condition on an event with probability 0 is the only thing to do: given that a d6 comes up both odd and even, what is the probability that the result is higher than 3?

[I tried saying some things about conditioning on sets of measure 0 here, but apparently I don't know what I'm talking about so I will retract that portion of the comment for the sake of clarity.]

Comment author: 06 July 2012 01:22:02AM 2 points [-]

In more mathematical settings, you can successfully condition on events with probability 0 (for instance, if (X,Y) follow a bivariate normal distribution, you might want to know the probability distribution of Y given X=x).

You can't really do this, since the answer depends on how you take the limit. You can find a limit of conditional probabilities, but saying "the probability distribution of Y given X=x" is ambiguous. This is known as the Borel-Kolmogorov paradox.

Comment author: 06 July 2012 01:29:19AM 2 points [-]

Oops. Right, I knew there were some problems here, but I thought the way I defined it I was safe. I guess not. Thanks for keeping me honest!

Comment author: 05 July 2012 04:15:26PM *  2 points [-]

Log-odds are perfectly isomorphic with probabilities and satisfies Cox's Theorem. Saying that log-odds are not what probabilities are is as non-sequiteur as saying 2+2 isn't a valid representation of 4.

Bayes theorem assumes no such thing as non-zero probability, it assumes Real Numbered probabilities, as it is in fact a perfectly valid statement of real-number arithmetic in any other context. It just so happens to be that this arithmetic expression is undefined for when certain variables are 0, and is an identity (equal to 1) when certain variables are 1. Neither are particularly interesting.

Bayes Theorem is interesting because it becomes propositional logic when you apply it to a limit going towards 1 or 0.

Real life applications are not my expertise, but I know my groups, categories and types. 0 and 1 are not probabilities, just as positive and negative infinity are not Real Numbers. This is a truth derived directly from Russel's Axioms, which is the definition basis for all modern mathematics.

When you say P(A) = 1 you are not using probabilities anymore, At best your are doing propositional logic, at worst you'll get a type error. If you want to be as sure as you can, let credence be 1 - epsilon for arbitrarily small positive real epsilon.

Comment author: 05 July 2012 06:27:08PM 4 points [-]

Clearly log-odds aren't perfectly isomorphic with multiplicative probabilities, since clearly one allows probabilities of 0 and 1 and the other doesn't.

Bayes's theorem does assume nonzero probability, as you can observe by examining its proof.

1. Pr[A & B] = Pr[B] Pr[A|B] = Pr[A] Pr[B|A] by definition of conditional probability.
2. Pr[A|B] = Pr[A] Pr[B|A] / Pr[B] if we divide by Pr[B]. This assumes Pr[B]>0 because otherwise this operation is invalid.

You can't derive properties of probability from Russell's axioms, because these describe set theory and not probability. One standard way of deriving properties of probability is via Dutch Book arguments. These can only show that probabilities must be in the range [0,1] (including the endpoints). In fact, no finite sequence of bets you offer me can distinguish a credence of 1 from a credence of 1-epsilon for sufficiently small epsilon. (That is, for any epsilon, there's a bet that distinguishes 1-epsilon from 1, but for any sequence of bets, there's an 1-epsilon that is indistinguishable from 1).

Here is an analogy. The well-known formula D = RT describes the relationship between distance traveled, average speed, and time. You can also express this as log(D) = log(R) + log(T) if you like, or D/R = T. In either of these formulas, setting R=0 will be an error. This doesn't mean that there's no such thing as a speed of 0, and if you think your speed is 0 you are actually traveling at a speed of epsilon for some very small value of epsilon. It just means that when you passed to these (mostly equivalent) formulations, you lost the capability to discuss speeds of 0. In fact, when we set R to 0 in the original formula, we get a more useful description of what happens: D=0 no matter the value of T. In other words, 0 is a valid speed, but you can't travel a nonzero distance with an average speed of zero, no matter how much time you allow yourself.

What is the difference between log-odds and log-speeds, that makes the former an isomorphism and the latter an imperfect description?

Finally, do you really think that someone who thinks "0 and 1 are probabilities" is a statement LW is irrational about is unaware of the "0 and 1 are not probabilities" post?

Comment author: 06 July 2012 12:54:35PM 0 points [-]

Potholing that last sentence was mostly for fun.

By virtue of the definition of a logarithm, exp(log(x)) = x, we can derive that since the exponential function is well-defined for complex numbers, so is the logarithm. Taking the logarithm of a negative number nets you the logarithm of the absolute plus imaginary pi. The real part of any logarithm is a symmetric function, and there are probably a few other interesting properties of logarithms in complex analysis that I don't know of.

log(0) is undefined, as you note, but that does not mean the limit x -> 0 log(x) is undefined. It is in fact a pole singularity (if I have understood my singularity analysis correct). No matter how you approach it, you get negative infinity. So given your "logarithmic velocities," I counter with the fact that using a limit it is still a perfect isomorphism. Limit of x -> -inf exp(x) is indeed 0, so when using limits (which is practically what Real Numbers are for) your argument that log isn't an isomophism from reals to reals is hereby proven invalid (if you want a step by step proof, just ask, it'll be a fun exercise).

Given that logarithms are a category-theoretic isomorphism from the reals onto the reals (from the multiplication group onto the addition group) there is no reason why log-odds isn't as valid as odds, which is as valid as ]0,1[ probabilities. Infinities are not valid Reals, 0 and 1 are not valid probablities QED.

As I said. Do not challenge anyone (including me) on the abstract algebra of the matter.

[I do apologize if the argument is poorly formulated, I am as of writing mildly intoxicated]

Comment author: 08 July 2012 05:21:52PM 1 point [-]

Yes, log(x) is an isomorphism from the positive reals as a multiplicative group to the real numbers as an additive group. As a result, it is only an isomorphism from multiplicative probabilities to additive log-probabilities if you assume that 0 is not a probability to begin with, which is circular logic.

Comment author: 08 July 2012 05:44:27PM 0 points [-]

So, pray tell: When are P=0 and P=1 applicable? Don't you get paradoxes? What prior allows you to attain them?

I am really genuinely curious what sort of proofs you have access to that I do not.

Comment author: 03 July 2012 11:34:14PM 1 point [-]

Does this belief really affect anything, or is it only a proposition considered true without any consequences on your cognitive processes? (I've always regarded "0 and 1 are not probabilities" as more of a rhetorical figure than a statement of belief.)

Comment author: 04 July 2012 12:10:30AM *  2 points [-]

Well, on a somewhat trivial note, I (plan to) make my living proving that certain things have probabilities distinct from 0, so if 0 and 1 weren't probabilities to begin with I'd be out of a job.

That's not really it, though, because I think the "0 and 1 are not probabilities" claim is really about degrees of belief in non-mathematical propositions. In its most-reasonable-to-me form, it says something like "Even if you have an argument that statement S is true with probability 1, you should believe Pr[S] < 1, because your argument could be wrong". And there's... really not a lot I could say in response to that. Except I would note that the value 1 isn't really special here.

But there's a lot of things that go together with this idea that I do disagree with. In very many senses, even non-mathematical propositions do end up having probabilities of 0 or 1. For instance:

• Any time we deal with (even theoretical) infinities (this one is important because here we get events with probability 0 that can actually happen)
• Tautologies (duh)
• Conditional probabilities (nobody really disagrees with this, but I think lots of probabilities we think are unconditional aren't)
• Any belief that I can never be talked out of (given how the human mind works, probably most beliefs we have are like this actually)

Plus in practice accepting "0 and 1 are not probabilities" rhetorically or otherwise just means that you stop writing 1 and start writing 1-epsilon. Whose belief is it really that doesn't affect anything?

Comment author: 04 July 2012 09:59:59PM 1 point [-]

even non-mathematical propositions do end up having probabilities of 0 or 1 ... Tautologies

Tautologies are true for mathematical reasons and there is little difference - as far as probability assessment goes - between "P ∨ ∽P" and "Yding Skovhøj is the highest peak of Egypt or Yding Skovhøj is not the highest peak of Egypt". Thus, tautologies (and pseudologies, or how do we call their false counterparts) don't really make a category distinct from mathematical statements.

Conditional probabilities

I am not sure what you mean here. Of course there are conditional probabilities of form "if X, then X", but they already belong to the tautology group.

Regarding mathematical statements it's nevertheless important to notice that there are two meanings of "probability". First, there is what I would call "idealised" or "mathematical probability", formally defined inside a mathematical theory. One typically defines probability as a measure over some abstract space and usually is able to prove that there exist sets of probability 1 or 0. This is, more or less, the sort of probability relevant to the probabilistic method you have linked to. Second, there is the "psychological" probability which has the intuitive meaning of "degree of belief", where 1 and 0 refer to absolute certainty. This is, more or less, the sort of probability spoken about in "1 and 0 aren't probabilities".

These two kinds of probabilities may correspond to each other more or less closely, but aren't the same: having a formal proof of a proposition isn't the same as being absolutely certain about it; people make mistakes when checking proofs.

Plus in practice accepting "0 and 1 are not probabilities" rhetorically or otherwise just means that you stop writing 1 and start writing 1-epsilon. Whose belief is it really that doesn't affect anything?

If believing P doesn't affect anything, then naturally believing non-P doesn't affect anything either. So, if you agree that "1 and 0 aren't probabilities" is an inconsequential belief, does it mean that your answer to my original question is "yes"?

Comment author: 04 July 2012 11:28:49PM 0 points [-]

If believing P doesn't affect anything, then naturally believing non-P doesn't affect anything either. So, if you agree that "1 and 0 aren't probabilities" is an inconsequential belief, does it mean that your answer to my original question is "yes"?

Saying that my belief in P is inconsequential implies that actually I am acting as if I believed not-P, even though I profess a belief in P. I argue that, conversely, many people who profess a belief in not-P act as if they actually believe P.

Comment author: 05 July 2012 01:41:31PM 1 point [-]

The point is that "acting as if one believes P" and "acting as if one believes not-P" can sometimes be the same actings. This is what I meant by "inconsequential". I want to know whether, in your opinion, this is such a situation; that is, whether there is some imaginable behaviour (other than professing the belief) which would make sense if one believed that "1 is not a probability" but would not make sense if one believed otherwise.

Comment author: 04 July 2012 03:51:20AM *  1 point [-]

Any belief that I can never be talked out of (given how the human mind works, probably most beliefs we have are like this actually)

I suspect that with enough resources you could be talked out of any of your beliefs. Oh, sure, it would take a lot of time, planning, and manpower (and probably some people you approve of having the beliefs we'd want to indoctrinate you with). You're not actually 100% certain that you're 100% certain that 0 and 1 are probabilities.

The trouble with thinking 0 or 1 is a probability is that it is exactly equivalent to having an infinite amount of evidence, which is impossible by the laws of thermodynamics; minds exist within physics.

Furthermore, a feeling of absolute certainty isn't even a number, much less a probability.

Comment author: 04 July 2012 07:13:48PM *  1 point [-]

I suspect that with enough resources you could be talked out of any of your beliefs.

At some point you have to ask: who is this "me" that can have any arbitrary collection of beliefs?

(And yes, incidentally, I don't assign 100% probability to the fact that I assign 100% probability to the statement "0 and 1 are probabilities." I think I could be persuaded, not to have a lower confidence in the 0-1 statement, but to believe that my confidence in it is lower than it is. This is sort of hard to think about, though.)

Comment author: 27 November 2012 03:54:45AM 0 points [-]

Funny.

Comment author: 06 July 2012 05:13:51AM 0 points [-]

This doesn't offer any anticipation about the world for me to agree or disagree with. Probability is just a formalism you use, and there's no reason for you to not define the formalism anyway you want.

Comment author: 04 July 2012 01:11:53PM -1 points [-]

Downvoted for agreement. Of course usually it isn't rational to assign probabilites of 0 and 1, but in this case I think it is.

Comment author: 04 July 2012 10:18:41AM -1 points [-]

I'd not seen Elizier's post on "0 and 1 are not probabilities" before. It was a very interesting point. The link at the end was very amusing.

However, it seems he meant "it would be more useful to define probabilities excluding 0 and 1" (which may well be true), but phrased it as if it were a statement of fact. I think this is dangerous and almost always counterproductive -- if you mean "I think you are using these words wrong" you should say that, not give the impression you mean "that statement you made with those words is false according to your interpretation of those words is false".

Comment author: 03 July 2012 11:06:42PM -1 points [-]

Upvoted in furious, happy disagreement, because I was going to post this very thing, with a confidence level of 20%, but then I reasoned out that this was unbelievably stupid and the probability of infinite Bayesian evidence being possible should be the same as probabilities for other things we have very strong reason to believe are simply impossible: 1 - epsilon.

Comment author: 03 July 2012 11:19:07PM 2 points [-]

should be the same as probabilities for other things we have very strong reason to believe are simply impossible: 1 - epsilon.

I'm pretty sure the probability of almost certainly impossible things being possible is lower than 1-epsilon. Except for very large values of epsilon.

Comment author: 04 July 2012 02:03:59AM 2 points [-]

Indeed, for values of epsilon approaching one.

Comment author: 03 July 2012 11:09:49PM 1 point [-]

I suppose if I wanted to maximize karma I should have stated a confidence level of 0%.

Comment author: 03 July 2012 11:14:42PM 7 points [-]

You're supposed to post things you actually believe, you know! What are you, a spirit-of-the-game violator?

Comment author: 03 July 2012 11:19:08PM *  -1 points [-]

Gaining utility from karma, illegitemate or fraudulent sources regardless, is an ongoing problem which never ceases to amuse me. Let the humans have their fun!

Comment author: 03 July 2012 11:16:30PM 1 point [-]

I do believe the 100% thing, though. It's just that in this case, karma is not maximized where spirit-of-the-game is maximized, and I thought I'd point that out.