qsz comments on Open Thread, Apr. 27 - May 3, 2015 - Less Wrong

3 Post author: Gondolinian 27 April 2015 12:18AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (352)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 28 April 2015 09:54:42AM 2 points [-]

If you didn't have any control group, you wouldn't be able to interpret any improvement between pretest and posttest, if you observed such a pattern: repetition or practice effects could explain any improvement. If you observed no improvement, you wouldn't need a control group because there's no effect to be explained.

Sometimes exploratory methods start out with no-control group pilots just to see if a method is potentially promising (if no hints of effects, don't invest a lot of resources in trying to set up a proper study).

Sometimes studies like this are set up with multiple control groups to address specific concerns that may apply to individual control conditions. Here it seems like two would be the minimum: one in which participants play a different game that is expected to confer no benefit for learning; and another with some kind of more traditional instruction.

In cases like this, recruitment is usually very vague - giving participants a realistic impression of the kinds of tasks they will be asked to do, and definitely no indications about who is assigned to a "control" group.