Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

DanielVarga comments on Bayes' Theorem Illustrated (My Way) - Less Wrong

126 Post author: komponisto 03 June 2010 04:40AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (191)

You are viewing a single comment's thread.

Comment author: DanielVarga 03 June 2010 06:53:11AM 7 points [-]

Wonderful. Are you aware of the Tuesday Boy problem? I think it could have been a more impressive second example.

"I have two children. One is a boy born on a Tuesday. What is the probability I have two boys?"

(The intended interpretation is that I have two children, and at least one of them is a boy-born-on-a-Tuesday.)

I found it here: Magic numbers: A meeting of mathemagical tricksters

Comment author: ciphergoth 03 June 2010 10:14:07AM *  15 points [-]

I always much prefer these stated as questions - you stop someone and say "Do you have exactly two children? Is at least one of them a boy born on a Tuesday?" and they say "yes". Otherwise you get into wondering what the probability they'd say such a strange thing given various family setups might be, which isn't precisely defined enough...

Comment author: Emile 03 June 2010 01:17:35PM *  4 points [-]

Very true. The article DanielVarga linked to says:

If you have two children, and one is a boy, then the probability of having two boys is significantly different if you supply the extra information that the boy was born on a Tuesday. Don't believe me? We'll get to the answer later.

... which is just wrong: whether it is different depends on how the information was obtained. If it was:

-- On which day of the week was your youngest boy born ?

-- On a Tuesday.

... then there's zero new information, so the probability stays the same, 1/3rd.

(ETA: actually, to be closer to the original problem, it should be "Select one of your sons at random and tell me the day he was born", but the result is the same.)

Comment author: Christian_Szegedy 04 June 2010 12:09:44AM *  1 point [-]

I think the only reasonable interpretation of the text is clear since otherwise other standard problems would be ambiguous as well:

"What is probability that a person's random coin toss is tails?"

It does not matter whether you get the information from an experimenter by asking "Tell me the result of your flip!" or "Did you get tails?". You just have to stick to the original text (tails) when you evaluate the answer in either case.

[[EDIT] I think I misinterpreted your comment. I agree that Daniel's introduction was ambiguous for the reasons you have given.

Still the wording "I have two children, and at least one of them is a boy-born-on-a-Tuesday." he has given clarifies it (and makes it well defined under the standard assumptions of indifference).

Comment author: DanielVarga 04 June 2010 08:33:00AM *  6 points [-]

Yesterday I told the problem to a smart non-math-geek friend, and he totally couldn't relate to this "only reasonable interpretation". He completely understood the argument leading to 13/27, but just couldn't understand why do we assume that the presenter is a randomly chosen member of the population he claims himself to be a member of. That sounded like a completely baseless assumption to him, that leads to factually incorrect results. He even understood that assuming it is our only choice if we want to get a well-defined math problem, and it is the only way to utilize all the information presented to us in the puzzle. But all this was not enough to convince him that he should assume something so stupid.

Comment author: Christian_Szegedy 04 June 2010 06:03:52PM 3 points [-]

For me, the eye opener was this outstanding paper by E.T. Jaynes:

http://bayes.wustl.edu/etj/articles/well.pdf

IMO this describes the essence of the difference between the Bayesian and frequentist philosophy way better than any amount of colorful polygons. ;)

Comment author: ciphergoth 03 June 2010 02:27:58PM 1 point [-]

I get that assuming that genders and days of the week are equiprobable, of all the people with exactly two children, at least one of whom is a boy born on a Tuesday, 13/27 have two boys.

Comment author: Emile 03 June 2010 03:53:05PM *  1 point [-]

True, but if you go around asking people-with-two-chidren-at-least-one-of-which-is-a-boy "Select one of your sons at random, and tell me the day of the week on which he was born", among those who answer "Tuesday", one-third will have two boys.

(for a sufficiently large set of people-with-two-chidren-at-least-one-of-which-is-a-boy who answer your question instead of giving you a weird look)

I'm just saying that the article used an imprecise formulation, that could be interpreted in different ways - especially the bit "if you supply the extra information that the boy was born on a Tuesday", which is why asking questions the way you did is better.

Comment author: neq1 03 June 2010 02:49:38PM 1 point [-]

It seems to me that the standard solutions don't account for the fact that there are a non-trivial number of families who are more likely to have a 3rd child, if the first two children are of the same sex. Some people have a sex-dependent stopping rule.

P(first two children different sexes | you have exactly two children) > P(first two children different sexes | you have more than two children)

The other issue with this kind of problem is the ambiguity. What was the disclosure algorithm? How did you decide which child to give me information about? Without that knowledge, we are left to speculate.

Comment author: JenniferRM 03 June 2010 06:14:09PM *  4 points [-]

This issue is also sometimes raised in cultures where male children are much more highly prized by parents.

Most people falsely assume that such a bias, as it stands, changes gender ratios for the society, but its only real effect is that correspondingly larger and rarer families have lots of girls. Such societies typically do have weird gender ratios, but this is mostly due to higher death rates before birth because of selective abortion, or after birth because some parents in such societies feed girls less, teach them less, work them more, and take them to the doctor less.

Suppose the rules for deciding to have a child without selective abortion (and so with basically 50/50 odds of either gender) and no unfairness post-birth were: If you have a boy, stop; if you have no boy but have fewer than N children, have another. In a scenario where N > 2, two child families are either a girl and a boy, or two girls during a period when their parents still intend to have a third. Because that window is relatively small relative to the length of time that families exist to be sampled, most two child families (>90%?) would be gender balanced.

Generally, my impression is that parental preferences for one or the other sex (or for gender balance) are generally out of bounds in these kinds of questions because we're supposed to assume platonicly perfect family generating processes with exact 50/50 odds, and no parental biases, and so on. My impression is that cultural literacy is supposed to supply the platonic model. If non-platonic assumptions are operating then different answers are expected as different people bring in different evidence (like probabilities of lying and so forth). If real world factors sneak in later with platonic assumptions allowed to stand then its a case of a bad teacher who expects you to guess the password of precisely which evidence they want to be imported, and which excluded.

This issue of signaling which evidence to import is kind of subtle, and people get it wrong a lot when they try to tell a paradox. Having messed them up in the past, I think it's harder than telling a new joke the first time, and uses similar skills :-)

Comment author: Fyrius 07 June 2010 12:56:09PM *  0 points [-]

Note before you start calculating this: There's a distinction between the "first" and the "second" child made in the article. To avoid the risk of having to calculate all over again, take this into account if you want to compare your results to theirs.

I calculated the probability without knowing this, so that I just counted BG and GB as one scenario, where there's one girl and one boy. That means that without the Tuesday fact the probability of another boy is 1/2, not 1/3.

(I ended up at a posterior probability of 2/3, by the way.)

Comment author: Morendil 28 August 2010 10:12:05PM 1 point [-]

The Tuesday Boy was loads of fun to think about the first time I came across it - thanks to the parent comment. I worked through the calculations with my 14yo son, on a long metro ride, as a way to test my understanding - he seemed to be following along fine.

The discussion in the comments on this blog on Azimuth has, however, taken the delight to an entirely new level. Just wanted to share.

Comment author: JoshuaZ 03 June 2010 12:54:46PM 1 point [-]

Actually, a Bayesian and a frequentist can have different answers to this problem. It resides on what distribution you are using to decide to tell me that a boy is born on Tuesday. The standard answer ignores this issue.

Comment author: DanielVarga 03 June 2010 02:20:06PM 2 points [-]

I don't know much about the philosophy of statistical inference. But I am dead sure that if the Bayesian and the frequentist really do ask the same question, then they will get the same answer. There is a nice spoiler post where the possible interpretations of the puzzle are clearly spelled out. Do you suggest that some of these interpretations are preferred by either a frequentist or a Bayesian?

Comment author: JoshuaZ 03 June 2010 02:32:30PM 1 point [-]

Well, essentially, focusing on that coin flip is a very Bayesian thing to do. A frequentist approach to this problem won't imagine the prior coin flip often. See Eliezer's post about this here. I agree however that a careful frequentist should get the same results as a Bayesian if they are careful in this situation. What results one gets depends in part on what exactly one means by a frequentist here.

Comment author: bigjeff5 22 December 2013 01:18:46AM 0 points [-]

Just so it's clear, since it didn't seem super clear to me from the other comments, the solution to the Tuesday Boy problem given in that article is a really clever way to get the answer wrong.

The problem is the way they use the Tuesday information to confuse themselves. For some reason not stated in the problem anywhere, they assume that both boys cannot be born on Tuesday. I see no justification for this, as there is no natural justification for this, not even if they were born on the exact same day and not just the same day of the week! Twins exist! Using their same bizarre reasoning but adding the extra day they took out I get the correct answer of 50% (14/28), instead of the close but incorrect answer of 48% (13/27).

Using proper Bayesian updating from the prior probabilities of two children (25% boys, 50% one each, 25% girls) given the information that you have one boy, regardless of when he was born, gets you a 50% chance they're both boys. Since knowing only one of the sexes doesn't give any extra information regarding the probability of having one child of each sex, all of the probability for both being girls gets shifted to both being boys.

Comment author: Jiro 22 December 2013 03:41:40AM *  -1 points [-]

No, that's not right. They don't assume that both boys can't be born on Tuesday. Instead, what they are doing is pointing out that although there is a scenario where both boys are born on Tuesday, they can't count it twice--of the situations with a boy born on Tuesday, there are 6 non-Tuesday/Tuesday, 6 Tuesday/non-Tuesday, and only 1, not 2, Tuesday/Tuesday.

Actually, "one of my children is a boy born on Tuesday" is ambiguous. If it means "I picked the day Tuesday at random, and it so happens that one of my children is a boy born on the day I picked", then the stated solution is correct. If it means "I picked one of my children at random, and it so happens that child is a boy, and it also so happens that child was born on Tuesday", the stated solution is not correct and the day of the week has no effect on the probability

Comment author: bigjeff5 22 December 2013 04:16:49AM *  0 points [-]

No, read it again. It's confusing as all getout, which is why they make the mistake, but EACH child can be born on ANY day of the week. The boy on Tuesday is a red herring, he doesn't factor into the probability for what day the second child can be born on at all. The two boys are not the same boys, they are individuals and their probabilities are individual. Re-label them Boy1 and Boy2 to make it clearer:

Here is the breakdown for the Boy1Tu/Boy2Any option:

Boy1Tu/Boy2Monday Boy1Tu/Boy2Tuesday Boy1Tu/Boy2Wednesday Boy1Tu/Boy2Thursday Boy1Tu/Boy2Friday Boy1Tu/Boy2Saturday Boy1Tu/Boy2Sunday

Then the BAny/Boy1Tu option:

Boy2Monday/Boy1Tu Boy2Tuesday/Boy1Tu Boy2Wednesday/Boy1Tu Boy2Thursday/Boy1Tu Boy2Friday/Boy1Tu Boy2Saturday/Boy1Tu Boy2Sunday/Boy1Tu

Seven options for both. For some reason they claim either BTu/Tuesday isn't an option, or Tuesday/BTu isn't an option, but I see no reason for this. Each boy is an individual, and each boy has a 1/7 probability of being born on a given day. In attempting to avoid counting evidence twice you've skipped counting a piece of evidence at all! In the original statement, they never said one and ONLY one boy was born on Tuesday, just that one was born on Tuesday. That's where they screwed up - they've denied the second boy the option of being born on Tuesday for no good reason.

A key insight that should have triggered their intuition that their method was wrong was that they state that if you can find a trait rarer than being born on Tuesday, like say being born on the 27th of October, then you'll approach 50% probability. That is true because the actual probability is 50%.

Comment author: Jiro 22 December 2013 04:24:10AM 0 points [-]

Here is the breakdown for the Boy1Tu/Boy2Any option:

Boy1Tu/Boy2Tuesda

Then the BAny/Boy1Tu option:

Boy2Tuesday/Boy1Tu

You're double-counting the case where both boys are born on Tuesday, just like they said.

A key insight that should have triggered their intuition that their method was wrong was that they state that if you can find a trait rarer than being born on Tuesday, like say being born on the 27th of October, then you'll approach 50% probability.

If you find a trait rarer than being born on Tuesday, the double-counting is a smaller percentage of the scenarios, so being closer to 50% is expected.

Comment author: bigjeff5 22 December 2013 04:55:28AM *  0 points [-]

I see my mistake, here's an updated breakdown:

Boy1Tu/Boy2Any

Boy1Tu/Boy2Monday Boy1Tu/Boy2Tuesday Boy1Tu/Boy2Wednesday Boy1Tu/Boy2Thursday Boy1Tu/Boy2Friday Boy1Tu/Boy2Saturday Boy1Tu/Boy2Sunday

Then the Boy1Any/Boy2Tu option:

Boy1Monday/Boy2Tu Boy1Tuesday/Boy2Tu Boy1Wednesday/Boy2Tu Boy1Thursday/Boy2Tu Boy1Friday/Boy2Tu Boy1Saturday/Boy2Tu Boy1Sunday/Boy2Tu

See 7 days for each set? They aren't interchangeable even though the label "boy" makes it seem like they are.

Do the Bayesian probabilities instead to verify, it comes out to 50% even.

Comment author: shinoteki 22 December 2013 05:25:35AM *  0 points [-]

What's the difference between

Boy1Tu/Boy2Tuesday

and

Boy1Tuesday/Boy2Tu

?

Comment author: bigjeff5 22 December 2013 05:35:18AM 0 points [-]

In Boy1Tu/Boy2Tuesday, the boy referred to as BTu in the original statement is boy 1, in Boy2Tu/Boy1Tuesday the boy referred to in the original statement is boy2.

That's why the "born on tuesday" is a red herring, and doesn't add any information. How could it?

Comment author: Jiro 22 December 2013 05:51:45AM *  0 points [-]

This sounds like you are trying to divide "two boys born on Tuesday" into "two boys born on Tuesday and the person is talking about the first boy" and "two boys born on Tuesday and the person is talking about the second boy".

That doesn't work because you are now no longer dealing with cases of equal probability. "Boy 1 Monday/Boy 2 Tuesday", "Boy 1 Tuesday/Boy 2 Tuesday", and "Boy 1 Tuesday/Boy 1 Monday" all have equal probability. If you're creating separate cases depending on which of the boys is being referred to, the first and third of those don't divide into separate cases but the second one does divide into separate cases, each with half the probability of the first and third.

doesn't add any information. How could it?

As I pointed out above, whether it adds information (and whether the analysis is correct) depends on exactly what you mean by "one is a boy born on Tuesday". If you picked "boy" and "Tuesday" at random first, and then noticed that one child met that description, that rules out cases where no child happened to meet the description. If you picked a child first and then noticed he was a boy born on a Tuesday, but if it was a girl born on a Monday you would have said "one is a girl born on a Monday", you are correct that no information is provided.

Comment author: bigjeff5 22 December 2013 06:06:39AM *  0 points [-]

The only relevant information is that one of the children is a boy. There is still a 50% chance the second child is a boy and a 50% chance that the second child is a girl. Since you already know that one of the children is a boy, the posterior probability that they are both boys is 50%.

Rephrase it this way:

I have flipped two coins. One of the coins came up heads. What is the probability that both are heads?

Now, to see why Tuesday is irrelevant, I'll re-state it thusly:

I have flipped two coins. One I flipped on a Tuesday and it came up heads. What is the probability that both are heads?

The sex of one child has no influence on the sex of the other child, nor does the day on which either child was born influence the day any other child was born. There is a 1/7 chance that child 1 was born on each day of the week, and there is a 1/7 chance that child 2 was born on each day of the week. There is a 1/49 chance that both children will be born on any given day (1/7*1/7), for a 7/49 or 1/7 chance that both children will be born on the same day. That's your missing 1/7 chance that gets removed inappropriately from the Tuesday/Tuesday scenario.

Comment author: bigjeff5 22 December 2013 04:48:30AM *  0 points [-]

Which boy did I count twice?

Edit:

BAny/Boy1Tu in the above quote should be Boy2Any/Boy1Tu.

You could re-label boy1 and boy2 to be cat and dog and it won't change the probabilities - that would be CatTu/DogAny.

Comment author: ABranco 03 June 2010 12:34:18PM 0 points [-]

I'm so happy: I've just got this one right, before looking at the answer. It's damn beautiful.

Thanks for sharing.

Comment author: pjeby 03 June 2010 02:37:18PM 1 point [-]

I'm so happy: I've just got this one right, before looking at the answer. It's damn beautiful.

Same here. It was a perfect test, as I've never seen the Tuesday Boy problem before. Took a little wrangling to get it all to come out in sane fractions, and I was staring at the final result going, "that can't be right", but sure enough, it was exactly right.

(Funny thing: my original intuition about the problem wasn't that far off. I was simply ignoring the part about Tuesday, and focusing on the prior probability that the other child was a boy. It gives a close, but not-quite-right answer.)