Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Individual Deniability, Statistical Honesty

43 Post author: Alicorn 09 August 2011 04:17AM

If you have a lot of people to question about something, and they have a motivation to lie, consider this clever use of a six-sided die.

If the farmer tossed the die and got a one, they had to respond "yes" to the surveyor's question. If they got a six, they had to say "no." The rest of the time, they were asked to answer honestly. The die was hidden from the person who was conducting the survey, so they never knew what number the farmer was responding to.

Suddenly, the number of "yes" responses to the leopard question started coming up by more than just one-sixth.

Comments (8)

Comment author: Tesseract 09 August 2011 09:18:06AM *  13 points [-]

The keywords here are "randomized response". There are some interesting variations (from the Wikipedia page):

The sensitive question is worded in two dichotomous alternatives, and chance decides, unknown to the interviewer, which one is to be answered honestly.

Alternative 1: "I have consumed marijuana." Alternative 2: "I have never consumed marijuana." The interviewed are asked to secretly throw a die and answer the first question only if they throw a 6, otherwise the second question.

Comment author: MatthewBaker 09 August 2011 05:18:13PM 6 points [-]

I cant believe its that simple but in retrospect its obvious.

Comment author: [deleted] 09 August 2011 08:00:55PM 4 points [-]
Comment author: MatthewBaker 09 August 2011 09:13:08PM 4 points [-]

Agreed :) though i think its more the specific method that's so ingenious. Every anonymous survey ive ever taken has always told me at least 3 times it was anonymous but plausible dependability works so much better in hindsight.

Comment author: Tesseract 09 August 2011 09:06:46PM 5 points [-]

Also, it occurs to me that this is essentially an application of Bayes' Theorem. In an ordinary survey, the posterior probability (killed leopard|says yes) is 1, which is bad for the farmers, so they lie and therefore decrease the conditional probability (says yes|killed leopard), which is bad for the surveyors. Adding the die roll increases the unconditional probability of saying yes, so that the posterior probability no longer equals the conditional, and they can both get what they want.

Comment author: realitygrill 05 April 2012 05:56:15AM 1 point [-]

I reinvented this method, except using a coin flip, during an urban economics class. We were going to actually conduct surveys in the Detroit area and thus had to learn about the problems with surveys. However my professor didn't seem very excited and pointed out some incentive shortcomings (I was too disappointed to listen too closely), and I got the impression that this is a known method with known limitations.

Comment author: TraderJoe 02 March 2012 06:41:57PM *  0 points [-]

[comment deleted]

Comment author: MugaSofer 21 January 2013 04:54:32PM *  -2 points [-]

Suddenly, the number of "yes" responses to the leopard question started coming up by more than just one-sixth.

I fear any actual "leopard question" will not be as good as the ones that I imagined on reading that sentence.

Especially as the survey was about poaching in African countries :(