You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

DanielLC comments on This is why we can't have social science - Less Wrong Discussion

36 Post author: Costanza 13 July 2014 09:04PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (82)

You are viewing a single comment's thread. Show more comments above.

Comment author: DanielLC 28 August 2014 07:01:10PM 0 points [-]

Is there only one kind of test? Couldn't they find another proxy?

I'm only arguing that direct replication isn't a better use of resources than just going on to a followup experiment with a different methodology

If the followup is testing the same thing with a different methodology, then the metaphor works. If you run followup experiments just to find more detail, it would be like someone testing positive for cancer so then you run a test for what kind of cancer. You're assuming they have cancer when you run the second test, so the results could be misleading.

Comment author: someonewrongonthenet 28 August 2014 07:15:08PM *  0 points [-]

If the followup is testing the same thing with a different methodology, then the metaphor works.

Generally an idea is considered well supported when multiple methodologies support it, yes. In the psychology lab I used to work in, at least, we never try to replicate, but we do try to show the same thing in multiple different ways. There are maybe 15 different experiments a year, but they're generally all centered around proving or dis-proving a cluster of 2 or 3 broad, conceptually linked hypotheses.

Biology labs I've worked with do often do the whole "okay, the results are in and this is established now, let's find additional detail' thing, but that's because they were usually looking at much simpler systems, like a single protein or something, so they could afford to take liberties and not be so paranoid about experimental methods.