= 27d567bc2d48d9f40e9ccc20ea370b15)
DanielLC comments on This is why we can't have social science - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (82)
If you get a positive result, you run another test. If you keep getting positive results, you probably have breast cancer.
Similarly, if an experiment fails to replicate, you try again. If it replicates this time, then it's probably fine. If it keeps failing to replicate, then there's a problem.
At the very least, you need to try to replicate a random sample of studies, just to make sure there aren't more false studies than you've been assuming.
Not an expert on cancer, but I don't think it works that way .I think the cancer test accurately tests a variable wihch is a proxy for cancer risk. So a patient who doesn't have cancer but tests positive will continue testing positive, because the variable that the cancer test measures as a proxy for cancer is elevated in that patient.
Experiments do work that way, but I'm not arguing against that. I'm only arguing that direct replication isn't a better use of resources than just going on to a followup experiment with a different methodology (unless direct replication is really easy and you can just have some students do it or something).
Is there only one kind of test? Couldn't they find another proxy?
If the followup is testing the same thing with a different methodology, then the metaphor works. If you run followup experiments just to find more detail, it would be like someone testing positive for cancer so then you run a test for what kind of cancer. You're assuming they have cancer when you run the second test, so the results could be misleading.
Generally an idea is considered well supported when multiple methodologies support it, yes. In the psychology lab I used to work in, at least, we never try to replicate, but we do try to show the same thing in multiple different ways. There are maybe 15 different experiments a year, but they're generally all centered around proving or dis-proving a cluster of 2 or 3 broad, conceptually linked hypotheses.
Biology labs I've worked with do often do the whole "okay, the results are in and this is established now, let's find additional detail' thing, but that's because they were usually looking at much simpler systems, like a single protein or something, so they could afford to take liberties and not be so paranoid about experimental methods.