Comment on Open Thread, Apr. 27 - May 3, 2015

Wed, 29 Apr 2015 11:57:45 GMT

I was thinking more about your previous posts on the subject (your development of the game and some of the ideas behind it). The same general reason I'd avoid testing people from my extended lab network, who may not know any details of a current study but have a sufficiently clear impression of what I'm interested in to potentially influence the outcomes (whether intentionally, "helping me out", or implicitly).

When rolling it out for testing, you could always include a post-test which probes people's previous experience (e.g. what they knew in advance about your work & the ideas behind it) & exclude people who report that they know "too much" about the motivations of the study. Could even prompt for some info about LW participation, could also be used to mitigate this issue (especially if you end up with decent samples both in and outside LW).

Discuss