= 27d567bc2d48d9f40e9ccc20ea370b15)
lavalamp comments on Case Study: Testing Confirmation Bias - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (61)
This was possibly an expensive experiment in terms of social capital...
I think it would have been better to have waited longer. After only three days, his response seems reasonable. Perhaps after two weeks, it would be more difficult to believe that he would have ever published your data.
He doesn't even tell us what the publication lag for the first experiment was.
The first experiment? You mean the SIAI habit formation thing? I thought it was obvious from the intro specifying when the call for applicants went up and when I posted, but I've edited it to be more explicit.
Or do you mean the vitamin D evening experiment? The results didn't contradict any of his theories, and to the extent it matters to the theory at all, his theory predicts that it ought to damage sleep in the evening since it's influencing circadian rhythms and it isn't a mere matter of vitamin D deficiency.
How long before he linked to your initial vitamin D results?
Dunno. As I said, it didn't matter.
It just occurred to me - I have an active experiment going with deleting random external links on Wikipedia, but even though this affects a rough minimum of ~335,445 readers of Wikipedia articles (based on the summed March statistics of the affected articles), I will probably catch far less flak when I post my results on the WikiEN-l mailing list than I have already caught for this post here. Humans!
I object to this more than I object to the experiment in the OP.
Bless your soul! I was completely disheartened at the disinterest of even Wikipedians in my earlier experiment demonstrating that suggestions for adding external links get ignored. Anger is better than apathy.
I agree, the number of people affected by an amateur experiment you perform is a good measure of how much flak you should catch.
On the other hand, people would be reading his site and drawing the wrong conclusions about D supplementation for two weeks. That's some further-spread epistemic pollution costs.
Yes, that was a major reason for only 3 days. Roberts makes it sound like he was going to do an in-depth analysis or whatever before discussing my data, but I don't believe this: if you look at the vitamin D category, you see he posts plenty of people's reports without formally analyzing their data but just describing it, and he had time to post something like 3 blog posts before I published this, one of which was a link roundup perfect for linking my results.
I didn't realize that people would see the 3 day waiting period as super-questionable. Thinking about it some more, I realize now what I should have done: I should have created a separate page on my site just for the fake results, and sent the subject that but linked it nowhere else. The subject would have no reason to be suspicious, the page would indeed be public, but it would not actually get any traffic from normal readers; hence, I could leave the fake page up for months.
(At some point I could even put up the real results on the main page (for the normal readers), since it would be unlikely for the subject to just randomly visit the page and notice the discrepancy.)