Unknown: What do we mean by "chance"? That it has a very small a priori probability... The evidence is given: the two sequences are similar. We can also assume that the evolution theory has a bigger probability a priori, than the chance to get that sequence. These insights were all included in the post, I think. So applying Bayes' theorem we get the fact that the evolution version has much bigger a posteriori probability, so we don't have to show that separately.

There are a lot of events which have a priori probabilities in that order of magnitude... But we also should have strong evidences to shift that to a plausible level. But a lot of people think: "there was only a very little chance for this to happen, but it happened => things with very little chances do happen sometimes."

James,

I agree that there's a difference between rhetoric and pure maths. However, you can change models, you can revise probabilities, update beliefs and argue the toss all day, but it doesn't make humans and chimps any less related, any more than you can argue the grass red. It's a good example about Kelvin's model of the sun. However, for it to be applicable, please tell us the chance that some future discovery will demonstrate that the similarities between human and chimp genomes are just a coincidence. See Eliezer's reference to 19th vs 21st century in the post.

The statement 'there's still a chance, right?' is mathematically valid in pretty much every case. The statement 'humans are genetically related to chimps' is rhetoric, and not any sort of Technical Argument in and of itself. However, I know which of these two has more relevance and meaning for me.

"The odds of that are something like two to the power of seven hundred and fifty million to one."

As Eliezer admitted, it is a very bad idea to ascribe probabilities like this to real world propositions. I think that the strongest reason is that it is just too easy for the presuppositions to be false or for your thinking to have been mistaken. For example, if I gave a five line logical proof of something, that would supposedly mean that there is no chance that its conclusion is true given the premisses, but actually the chance that I would make a logical error (even a transcription error somewhere) is at least on in a billion (~ 1 in 2^30). There is at least this much chance that either Elizer's reasoning or the basic scientific assumptions were seriously flawed in some way. Given the chance of error in even the simplest logical arguments (let alone the larger chance that the presuppositions about genes etc are false), we really shouldn't ascribe probabilities smaller than 1 in a billion to factual claims at all. Better to say that the probability of this happening by chance given the scientific presuppositions is vanishingly small. Or that the probability of it happening by chance pretty much equals the probability of the presuppositions being false.

Toby.

Toby,

What if there are more than a billion known options?

Carl, that is a good point. I'm not quite sure what to say about such cases. One thing that springs to mind though, is that in realistic examples you couldn't have investigated each of those options to see if it was a real option and even if you could, you couldn't be sure of all of that at once. You must know it through some more general principle whereby there is, say, an option per natural number up to a trillion. However, how certain can you be of that principle? That is isn't really up to only a million?

Hmmmm... Maybe I have an example that I can assert with confidence greater than one minus a billionth:

'The universe does not contain precisely 123,456,678,901,234,567,890 particles.'

I can't think of a sensible, important claim like Eliezer's original one though, and I stand by my advice to be very careful about claiming less than a billionth probability of error, even for a claim about the colour of a piece of paper held in front of you.

Toby.

Ben Jones: "The statement 'there's still a chance, right?' is mathematically valid in pretty much every case."

Exactly. The best answer would have been something like: "There's stil a chance for everything. There is no such thing as zero probabilities in the real world. Maybe I can cure everybody's cancer by wishing for it very, very hard. Sure, this thought violates everything we know about physics, but there is still a chance, no?"

Toby, I think that's a very good point. There is a difficulty in analyzing cases which involve a very large number of alternatives, such as my example of the number selected with odds of one in a google. But I think this difficulty is much like the difficulty of discussing the odds that 2 and 2 make 5; surely this cannot be assigned a probability of zero, and yet if it is assigned any positive probability, you can easily argue that it has a probability of unity.

I think the way to deal with this is to say that a statement can have an indefinitely small calculated probability, but on a human level there is a limit much as you stated, and this should be applied in retrospect even to our calculated probabilities; i.e. even though there is a calculated probability of one in a google that the particular sequence I posted above could be generated by chance, there is a human probability of at least one in a billion that all of my calculations are wrong anyway.

This is one reason why many of Eliezer's claims are overconfident: he seems to identify a calculated probability, or what he supposes the calculated probability would be if there was one, with a human probability.

But I think this difficulty is much like the difficulty of discussing the odds that 2 and 2 make 5; surely this cannot be assigned a probability of zero, and yet if it is assigned any positive probability, you can easily argue that it has a probability of unity.

Given certain assumptions, we can easily assign that a probability of zero.

The problems arise when we lose sight of the fact that we made assumptions.

But what I found even more fascinating was the qualitative distinction between "certain" and "uncertain" arguments, where if an argument is not certain, you're allowed to ignore it. Like, if the likelihood is zero, then you have to give up the belief, but if the likelihood is one over googol, you're allowed to keep it.

I think that's exactly what's going on. These people you speak of who do this are mentally dealing with social permission, not with probability algebra. The non-zero probability gives them social permission to describe it as "it might happen", and the detail that the probability is 1 / googolplex stands a good chance of getting ignored, lost, or simply not appreciated. (Similarly, the tiny uncertainty)

And I don't just mean that it works in conversation. The person who makes this mistake has probably internalized it too.

It struck me that way when I read your opening anecdote. Your interlocutor talked like a lawyer who was planning on bringing up that point in closing arguments - "Mr Yudkowsky himself admitted there's a chance apes and humans are not related" - and not bringing up the minuscule magnitude of the chance, of course.

It seems like it should be impossible to calculate a fudge factor into your calculations to account for the possibility that your calculations are totally wrong, because once you calculate it in it becomes part of your calculations, which could be totally wrong. Maybe I'm missing something here that would become apparent if I actually sat down and thought about the math, so if anybody has already thought about the math and can save me the time, I would appreciate it.

Nominull: It seems like it should be impossible to calculate a fudge factor into your calculations to account for the possibility that your calculations are totally wrong, because once you calculate it in it becomes part of your calculations, which could be totally wrong.

But wrong in which direction? If you don't know, it cancels out of the expectation of the probability. You just have to achieve a state where your meta-uncertainty seems balanced between both sides. Don't worry about justifying it to anyone, and particularly not justifying it to an ideal philosopher of perfect emptiness. Just give it your honest best shot, as a guesser.

In your friend's defense, I could turn that around:

Years ago, I was speaking to someone when he casually remarked that he believed in evolution. And I said, "This is not the nineteenth century. When Darwin first proposed evolution, it might have been reasonable to believe in it. But this is the twenty-first century. We can look at cells. Cells are hideously complex. It's over."

He said, "Maybe all the features arose by coincidence."

I said, "The odds of that are something like two to the power of seven hundred and fifty million to one."

He said, "But there's still a chance, right?"

Unknown, I agree entirely with your comments about the distinction between the idealised calculable probabilities and the actual error prone human calculations of them.

Nominull, I think you are right that the problem feels somewhat paradoxical. Many things do when considering actual human rationality (a species of 'bounded rationality' rather than ideal rationality). However, there is no logical problem with what you are saying. For most real world claims, we cannot have justifiable degrees of beliefs greater than one minus a billionth. Moreover, I don't have a justifiable degree of belief greater than one minus a billionth in my last statement being true (I'm pretty sure, but I could have made a mistake...). This lack of complete certainty about our lack of complete certainty is just one of the disadvantages of having resource bounds (time, memory, accuracy) on our reasoning. On a practical note, while we cannot completely correct ourselves, merely proposing a safe upper bound to confidence in typical situations, memorizing it as a simple number, and then using it in practice is fairly safe, and likely to improve our confidence estimates.

Toby.

Andy McKenzie -- that was my first thought too. Folks can view the scene here.

Eliezer, do you expect to be right more or less the next google times you calculate or estimate a probability of one in a google? If not, then for such probabilities we do know in which direction the estimate is likely to be mistaken, and so we can correct it, by the means suggested by Toby.

Rather that estimating a probability, it would have been more interesting to ask "What emotional need are you trying to meet with this?"

If Mr Still-a-chance yearns for "souls go to heaven and meet God" why does he care about evolution? Isn't the soul the magic, special sauce that converts an ordinary animal body into a human? How does denying evolution help him?

Meanwhile, 20000000 years in the future, a multi-generation interstellar space ship has set up a colony on a distant planet with existing biology. The colony collapses but man does not go extinct, and 100000 years later they have re-established a civilisation of sorts.

They find that man is not an animal. His biology is entirely distinct. Which goes well with their myths of a double fall, from the sky to the ground and from the golden age to barbarism, but what really do they gain when they find that they do not have genealogical ties to the animals around them. Why is our far future Mr Still-a-chance the 2nd so pleased?

I find myself unable to imagine how Mr Still-a-chance would have answered, which piques my curiosity

I think James Bach was on the right track here, but did not take this far enough. Eliezer's interlocuter was not able to really articulate his argument. Properly argued, probability is completely irrelevant.

So, let us contemplate the position of a serious, hard science creationist, and I hate to say it, but such people exist. So, this individual can fully agree and admit that how a given body grows and develops depends on its DNA structure, so that indeed it is not surprising that different species that appear morphologically and behaviorally to be somewhat similar, such as various canine species or feline species, or for that matter chimps and humans, even if the older creationists got all in fits about having a monkey for an uncle, and so forth, will have very similar DNA structures.

The issue then is how did this come to be. The evolutionist says that it is due to evolution from common ancestors and so forth. The scientific creationist says, "no," this simply reflects that the intelligent designer set them up this way because DNA controls the growth of individual entities, so similar appearing and behaving species will have more similar DNA, and God (or The Intelligent Designer) made it this way fully consciously in accord with the laws of science, which presumably the same Entity is also fully aware of, whether or not this Entity in fact set up those laws his or herself.

Eliezer: While you didn't specifically say that the guy you were arguing with was a creationist...... The creationists I find myself arguing with wouldn't say that the chimp and human DNA is similar by coincidence. My creationists would say that that the DNA is similar because:

1) DNA is what God used to program characteristics into living things. 2) God decided to make chimpanzees and humans similar. 3) To make them similar, He gave them similar DNA.

Eliezer: hey. how would your response change if arguing with these guys? also, you're awesome. just thought I would let you know.

Suzie's suspicion is correct. According to Bayes's theorem, the probability that he pulled it out randomly would be .05 x prior probability that he would pull out a marble randomly / prior probability that he would be found holding the red marble.

In this case it is rather difficult to calculate an exact number. But in Eliezer's case, an exact number is unnecessary; the ".05" in his case is so low that he assumes that the exact number will also be low, regardless of the particular values assigned to "prior probability of pulling out a marble randomly" and "prior probability that he ends up with a red marble."

Dan, you want An Intuitive Explanation of Bayes's Theorem.

When people say "every little bit counts!", I try to argue, "yeah, but only a little." Folks concern themselves with possibility when they should be concerned with probability; with existence when they should be concerned with magnitude.

Time for nitpicking... "Consider his example if you ever you find yourself thinking, “But you can’t prove me wrong.” If you’re going to ignore a probabilistic counterargument, why not ignore a proof, too?" - ...your own argument of certainty being infinity. In cardinal numbers theory the highest infinity (be it aleph-zero or continuum or 2^continuum or whatever) trumps any lower numbers (you can through out all the rational numbers, whose number is aleph-zero, and [0;1] will still be continual), including all natural numbers, and only an infinity of the same size or larger may compete. And I believe that the usual, single-infinity models do the same. If we _could_ have infinite certainty, it would be end-of-story, allowing for no possibility to "put the weight down - yes, down". The problem is, we can't.