gwern comments on Doing Important Research on Amazon's Mechanical Turk? - Less Wrong

5 Post author: peter_hurford 25 September 2013 05:04PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (49)

You are viewing a single comment's thread. Show more comments above.

Comment author: gwern 25 September 2013 09:19:52PM *  6 points [-]

I'd disagree; the 2% only come from an absurd overreading:

and 45 people reported, for example, that their chicken consumption decreased “slightly” or “significantly”.

So in the context of winning a contest with clear demand expectations and going only on cheap talk, without any measure of persistency over time, you only get 2% by counting anyone who claimed to be affected however 'slightly'? I think the more honest appraisal of that little experiment would be '0%'.

Comment author: peter_hurford 26 September 2013 01:52:41AM 0 points [-]

I think debating the merits of this particular percentage is not relevant enough to my topic to discuss further here. If you think that I am in error (or, worse, actively trying to manipulate the data to make my case look good), we could continue this conversation via PM or on a more relevant thread.