Matt_Simpson comments on Rationality Market Research - Less Wrong

59 Post author: Raemon 14 July 2011 07:41PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (86)

You are viewing a single comment's thread.

Comment author: Matt_Simpson 14 July 2011 09:03:10PM 17 points [-]

Just a note about analyzing this sort of data: almost anything you do can affect responses one way or another. The inflection of your voice, facial expressions, etc, let alone who is asking the subject. I've helped run an economic experiment where we recruited subjects in a shopping mall. Both the person who recruited the subject (we had two recruiters) and the person who read the instructions (we had two instructors as well, distinct from the recruiters) were significantly associated with the response, both more so than any other covariate.

I don't bring this up to say that what your doing is worthless, just to say be careful what conclusions you draw, especially if you don't have a standardized procedure across all subjects.

Comment author: Raemon 14 July 2011 09:45:15PM 0 points [-]

Agreed.