gwern comments on Value of Information: 8 examples - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (43)
As it happens, I learned how to do basic power calculations not that long ago. I didn't do an explicit calculation for the melatonin trial because I didn't randomize selection, instead doing an alternating days design and not always following that, so I thought why bother doing one in retrospect?
But if we were to wave that away, the power seems fine. I have something like 141 days of data, of which around 90-100 is usable, giving me maybe <50 pairs? If I fire up R and load in the two means and the standard deviation (which I had left over from calculating the effect size), and then play with the numbers, then to get an 85% chance I could find an effect at p=0.01:
If I drop the p=0.01 for 0.05, it looks like I should have had a good shot at detecting the effect:
So, it's not great, but it's at least not terribly wrong?
EDIT: Just realized that I equivocated over days vs pairs in my existing power analyses; 1 was wrong, but I apparently avoided the error in another, phew.