JonathanLivengood comments on Wisdom of the Crowd: not always so wise - Less Wrong

20 Post author: tgb 01 July 2012 08:55PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (20)
Tags: poll results publication bias

Comments (20)

You are viewing a single comment's thread.

Comment author: JonathanLivengood 01 July 2012 10:46:28PM 1 point [-]

Given a forced choice, I would have thought that you would want to look at the proportion of answers that were right -- in this case, right in the sense of being closer to the truth. When you look at the data that way, you see that people here were really good with respect to Africa (82% success rate) and really bad with respect to the history of the telephone (28% success rate).

What do you suppose would happen if you took a four-vector of two-choice options and presented all 16 vectors for a vote? How likely would we be to vote for the all-true vector out of the field of 16 choices?

Comment author: AlexMennen 02 July 2012 12:25:24AM 1 point [-]

I would have thought that you would want to look at the proportion of answers that were right -- in this case, right in the sense of being closer to the truth.

Closer to the truth than what?

Comment author: JonathanLivengood 02 July 2012 03:48:35AM 1 point [-]

Closer to the truth than the other answer. That is, if the truth is 1000 million and the available options are 1300 million and 200 million, then the first answer is 300 million away from the truth, while the second is 800 million away from the truth. One of those two answers is more approximately correct than the other.

Actually, looking back at the question asked, approximate truth isn't even an issue. The question asked which of the two numbers was closer to the (true) population of Africa. One is closer, the other is not. So, when one answers 1300 million in that forced choice, one is answering the question correctly.

Does that make sense?

Comment author: AlexMennen 02 July 2012 05:14:33AM 0 points [-]

Ah, I hadn't realized that people had been given only two options to choose from.

Comment author: Khoth 02 July 2012 06:59:50AM 0 points [-]

There were two versions of each question - one where you could answer anything, and one where there was a choice of two. I think the results here are from the free-choice version.

Comment author: Jayson_Virissimo 02 July 2012 03:14:50AM 0 points [-]

Closer to the truth than what?

Closer to the truth than coin flipping?

Powered by Reddit Powered by Reddit