othercriteria comments on Less Wrong Polls in Comments - Less Wrong

79 Post author: jimrandomh 19 September 2012 04:19PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (302)

Sort By: Popular

You are viewing a single comment's thread. Show more comments above.

Comment author: othercriteria 30 September 2012 03:34:09PM 3 points [-]

Your question is confused. The uniform distribution hypothesis only requires that the (assumed infinite) population picks the answers independently with equal probability. Under this hypothesis, the observed poll answers (for a fixed number of respondents) will follow a multinomial distribution with parameters (0.2, 0.2, 0.2, 0.2, 0.2). A typical realization will not have an equal number of respondents giving each answer, although asymptotically the empirical frequencies will converge to equality.

Anyways, as a Bayesian, the better question is what should my posterior belief about the response probabilities be after running the poll and updating off the answers? The canonical way to do this would be to put a Dirichlet prior over the response probabilities. By the miracle of conjugacy, your posterior distribution will itself by a (generally different) Dirichlet distribution.

By taking the expectation of indicator variables like I{"probability of First Answer under 0.2"} under the posterior, you can figure out what degree of belief you must give to statements like "respondents have an aversion toward First Answer".

Comment author: RobinZ 30 September 2012 03:40:20PM 3 points [-]

That makes sense - I had imagined doing something similar, but I had never heard of Dirichlet priors.

Comment author: othercriteria 30 September 2012 04:00:20PM 2 points [-]

Happy this helped. The Dirichlet-multinomial model gets relatively little attention because it adds nothing really new to the beta-binomial model for polls with just two responses. It's easy to find lots of introductory, chatty introductions to the beta-binomial like this one or this one if you want to learn more...