= 27d567bc2d48d9f40e9ccc20ea370b15)
Vaniver comments on Value of Information: 8 examples - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (43)
Sleep quality is typically measured by "number of times awakened" or "amount moved" or so on- my experience with melatonin (and I believe gwern's as well, but I didn't check) is that melatonin decreases the number of times I awaken during the night.
But even if you measure that, it's just a proxy. A paralytic drug will reduce the amount I toss and turn at night, but may not improve how I feel the next day. What you would want to do is measure energy level / creativity, but that's even more difficult.
I suppose measuring "blocks of time where I spent at least 30 minutes doing focused, productive work" would be an easy thing to correlate against having taken melatonin the night before, but I think that it would be very difficult to measure an effect with such low resolution data.
This is even easier if you use pomodoro, because measuring productive time and noticing unproductive time is much easier.
Interesting... I already use that exact method, but never heard that term for it. I first read about it in the "Now Habit" by Neil Fiore, a strategic system for overcoming procrastination. It works incredibly well!
Low-resolution doesn't really matter; what matters is how variable the data is. If you have a binary variable - as crude resolution as possible - which rarely flips, then an intervention which occasionally flips it will still be noticeable.
(How much data would it take? Well, that's hard to estimate without any data at all...)
That makes sense, I guess all of the other unknown variables would serve to dither the low resolution data and it would work for the same reason that CDs sound good with only 16 bits of resolution.
I am going to begin a random placebo controlled trial of melatonin use, quantified against my ZEO and work logs. I just need to find some opaque capsules.
I look forward to your results.
Thanks. The experiment starts today. I made up 14 pills and randomized them with R code, under the assumption that I couldn't possibly subconsciously track each pill and then remember their locations. In hindsight, this is probably more labor intensive and error prone than the method you used to randomize your adderall...
I start with a 14 tray pill box with treatment (0.75mg melatonin + parsely) in the first 7 boxes, and placebo (parsely pills only) in the last 7. I randomly reorder them 3 times, according to each of the lists outputted by my code, and it writes the final pill locations to a text file.
Here's my R code for randomizing "single blind" placebo controlled self-trials:
sampleSize <- 14
intialSetup <- c(rep("treatment", sampleSize/2), rep("placebo", sampleSize/2))
reorder1 <- sample(1:sampleSize, sampleSize, replace = FALSE)
reorder2 <- sample(1:sampleSize, sampleSize, replace = FALSE)
reorder3 <- sample(1:sampleSize, sampleSize, replace = FALSE)
final <- intialSetup[reorder1][reorder2][reorder3]
write.table(final, "final.txt")
paste(reorder1, "->", 1:sampleSize)
paste(reorder2, "->", 1:sampleSize)
paste(reorder3, "->", 1:sampleSize)
Interesting procedure. I'd agree it's probably much more work than some simple physical procedure. (I'd also point out that 14 pairs won't give you much significance - my above power analysis suggested that for awakenings, anyway, you'd want more like 140 pairs. But I should be happy you're actually doing the experiment.)
I plan to do much more than 14, but it was very tedious to set up so I started with that. I need to streamline the procedure.
How has the experiment been going?
Hey gwern, as you predicted I didn't have enough data to learn anything... and I didn't have time to do it longer. I considered repeating it, but now I'm scared off melatonin until I learn more about how it works. Dr. Ray Peat theorizes that it might have some negative health effects by inhibiting oxidative metabolism:
http://www.google.com/cse?cx=005233684413389937395%3Ad5qfhqsz7oo&ie=UTF-8&q=melatonin#gsc.tab=0&gsc.q=melatonin&gsc.page=1
Also, anecdotally I don't really see a huge benefit to melatonin. Even small doses (0.75mg) seem to make me slightly groggy when I wake 7.5 hours later. I may have unusually slow melatonin metabolism, as I have the "slow caffeine metabolizer" P450 CYP1A2 variant, the same enzyme responsible for clearing melatonin.
For my melatonin experiment, number of awakenings was 2.86 vs 2.43, but the p-value was only 0.43. The problem is that the standard deviation is 2.25! (On many nights, I awaken zero or one times, but on one particularly bad night, I woke up 7 times.) I suspect more data would show a more reliable effect and maybe a greater effect size than d=0.19.
To expand; if d<=0.19, to detect this effect at p<0.05 with 75% odds, we need ~75 pairs of nights or ~150 nights of data:
Doable, but not trivial.