wnoise comments on Navigating disagreement: How to keep your eye on the evidence - Less Wrong

37 Post author: AnnaSalamon 24 April 2010 10:47PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (72)
Tags: introduction practical disagreement

Comments (72)

You are viewing a single comment's thread. Show more comments above.

Comment author: AnnaSalamon 24 April 2010 10:50:03PM *  4 points [-]

Re: problem 1: Jelly bean number estimates are just like thermometer readings, except that the reading is in someone’s head, rather than their hand. So the obvious answer is to average everyone’s initial, solitary impressions, absent reason to expect one individual or another is an above-average (or below-average) estimator.

If your friends use lopsided weighting schemes in their second answers, should you re-update? This depends a lot on your friends.

  • Don't re-update from their answers if you think they don't understand the merits of averaging; you want to weight each person's raw impression evenly, not to overweight it based on how many others were randomly influenced by it (cf. information cascades: http://en.wikipedia.org/wiki/Information_cascade).
  • Do re-update if your friends understand the merits of averaging, such that their apparent over-weighting of a few peoples' datapoints suggests they know something you don't (e.g., perhaps your friend Julie has won past championships in jelly-bean estimation, and everyone but you knows it).
Comment author: cgm_E 26 April 2010 09:35:21PM *  0 points [-]

I would look for response clusters. Each participant could have a different counting method rendering different results (e.g. - estimate volumes/ count radius & height/ estimate there's an empty cone at the top which you don't see), and some methods could be common pitfalls. Therefore, some results - those obtained by a wrong way of counting, should be discarded, otherwise the median result would lead away from the right result. In order to decide which is the right response cluster, trying to figure out each method/mistake and determining the correct one would be useful. Of course, your method is not necessarily the right one, just because it's yours.

Powered by Reddit Powered by Reddit