PhilGoetz comments on Error detection bias in research - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (36)
This is a good point - and I would add that most of the errors that have this effect are not coding errors, but conceptual errors. A good practice, before performing an experiment, is to ask yourself, "What would I think if this just didn't work?" And try, hard, to come up with a justification for it not working. Sometimes I will realize there's an additional factor that may cause unexpected results.
Most of the time, though, I can't think how the experiment could fail until it actually fails. And then it seems obvious in retrospect why it failed.