qsz comments on Open Thread, Apr. 27 - May 3, 2015 - Less Wrong

3 Post author: Gondolinian 27 April 2015 12:18AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (352)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 29 April 2015 11:57:45AM 3 points [-]

I was thinking more about your previous posts on the subject (your development of the game and some of the ideas behind it). The same general reason I'd avoid testing people from my extended lab network, who may not know any details of a current study but have a sufficiently clear impression of what I'm interested in to potentially influence the outcomes (whether intentionally, "helping me out", or implicitly).

When rolling it out for testing, you could always include a post-test which probes people's previous experience (e.g. what they knew in advance about your work & the ideas behind it) & exclude people who report that they know "too much" about the motivations of the study. Could even prompt for some info about LW participation, could also be used to mitigate this issue (especially if you end up with decent samples both in and outside LW).

Comment author: Kaj_Sotala 01 May 2015 06:07:42PM 2 points [-]

Ah, that's a good point. And a good suggestion, too.