luispedro comments on Error detection bias in research - Less Wrong

54 Post author: neq1 22 September 2010 03:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (36)

You are viewing a single comment's thread.

Comment author: luispedro 25 September 2010 06:50:41PM 0 points [-]

Different fields have different standards. Some are more exacting than others and require full release of source code, have standard hidden-data competitions, have a culture of reviewing the software and attempting to reproduce &c (not all of the above is applicable to all fields). Others, not so much: you publish by giving a high level description of something you coded and people believe that you did it correctly and didn't spend hours looking for the parameters that gave you the prettiest graph. Debugging by "hacking 'til the graph is publishable" is, unfortunately, too common in some of those fields.

Many scientists are completely unaware of anything other than their field and will claim that "this is the it's done" whilst they only mean "this is the way that people in my narrow sub-field do it if they want to get published".