FrameBenignly comments on Open Thread April 4 - April 10, 2016 - Less Wrong

5 Post author: Elo 04 April 2016 04:56AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (211)

You are viewing a single comment's thread. Show more comments above.

Comment author: FrameBenignly 07 April 2016 06:06:17PM 0 points [-]

Good job on the study. I wouldn't call that a low n. I'm guessing you used a power analysis such as this one to calculate the sample size prior to the study, but the results are clearly wildly different from whatever your initial projections were, so the n you calculated prior to the study doesn't tell you all that much. Just eyeballing the covariance in your graph of the cumulative distribution (probability is usually listed on the y axis, btw), you've got a very strong result there. I do think intention-to-treat was the wrong system. I would have excluded all nights where you didn't want to use melatonin from the study, and randomly assigned nights where you were considering melatonin to either treatment or baseline. It looks like correcting for that would have given you even stronger results, however. But I think you went way past the point where more nights would have been of much use. Melatonin just doesn't appear to make much difference for you.