gwern comments on 2012 Less Wrong Census/Survey - Less Wrong

65 Post author: Yvain 03 November 2012 11:00PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (733)
Tags: meta survey

Comments (733)

You are viewing a single comment's thread. Show more comments above.

Comment author: ArthurDenture 04 November 2012 02:38:34PM *  18 points [-]

Took the survey.

I hope this question isn't used the way I worry it will be used:

CFAR Question 3

A certain town is served by two hospitals. In the larger hospital, about 45 babies are born each day. In the smaller one, about 15 babies are born each day. Although the overall proportion of girls is about 50%, the actual proportion at either hospital may be greater or less on any day. At the end of a year, which hospital will have the greater number of days on which more than 60% of the babies born were girls?

This question was easy for me to answer by pattern-matching to the Law of Small Numbers, as outlined in Thinking, Fast and Slow. If I hadn't read that, it's hard to say whether I would have reasoned it out correctly. So if many respondents answer this question correctly, I hope that the survey authors don't claim evidence that LW readers are better at statistical reasoning -- it'd be more accurate to say that LW readers are more likely to have seen this very particular question before.

(I could, naturally, be assuming too much about the intents of the survey authors.)

Comment author: gwern 04 November 2012 04:26:29PM 7 points [-]

This question was easy for me to answer by pattern-matching to the Law of Small Numbers, as outlined in Thinking, Fast and Slow. If I hadn't read that, it's hard to say whether I would have reasoned it out correctly. So if many respondents answer this question correctly, I hope that the survey authors don't claim evidence that LW readers are better at statistical reasoning -- it'd be more accurate to say that LW readers are more likely to have seen this very particular question before.

I don't understand the distinction you are making here. If you can answer correctly more statistical questions, how is that not being 'better at statistical reasoning'? Every area of thought draws heavily on memorization and caching.

Comment author: ArthurDenture 04 November 2012 06:43:41PM 7 points [-]

If you can answer correctly more statistical questions, how is that not being 'better at statistical reasoning'?

Those are related abilities, but there's being able to answer specific questions and then there's being able to apply what you've learned more generally. For me, this particular question triggered more "aha! I've seen this one before!" than it triggered statistical thought. A correct answer to the question might give you a smidgen of information on whether the answerer can reason about statistics, but it probably gives you a lot more information about whether the answerer has seen the question before.

One superficial example of dealing with this problem is how, in my college discrete math class, the professor gave us a problem involving placing pigeons in holes, with the solution having nothing to do with the pigeonhole principle. Even better than obfuscating a problem, of course, is stating a novel one that exercises the skills you're testing for.

Powered by Reddit Powered by Reddit