Meta-analysis on cognitive effects of modafinil (my bachelor thesis)
Well, meta-analyses certainly are an area of interest to me, and I was disappointed in 2012 by "Cognition Enhancement by Modafinil: A Meta-Analysis" (Kelley et al 20120) which used only 3 studies, and so was not very informative. A new meta-analysis would be great. But... I read quickly through it, and I saw no meta-analysis. Just a literature review. What's with the post title?
Modafinil significantly improved performance in 26 out of 102 cognitive tests, but significantly decreased performance in 3 cognitive tests.
Nitpick: I really hate this use of 'significantly' and I ban it from my own writing. Is this referring to effect sizes or p-values?
Notably, modafinil appears to have detrimental effects on mental flexibility. Although 4 studies employed the Intra/Extradimensional Set Shift task (ID/ED), no performance improvements could be detected. Performance was even reduced in a study by Randall et al. (2004). Furthermore, Müller et al. (2012) found that subjects on modafinil had lower flexibility scores in the Abbreviated Torrance task for adults.
Eh. Absence of improvement != damage. Randal 2004 didn't find a statistically-significant decrease (and it's not clear whether it should, given that it reports 25 datasets for 3 groups, so hunting for decreases incurs worries about multiplicity). And I have to point out, as far as Müller et al 2012 goes, the decrease didn't reach p<0.05 (just 0.053), and if you're willing to accept just trending, then you should also be accepting the increase in the GEFT/Group Embedded Figures Task (p=0.08).
How important are these observations...? Well, as you found out, it can be hard to compare or meta-analyze psychology studies since studies may cover the same topic but use different sets of tests, frustrating the most obvious approach 'just univariate meta-analyze everything!'
Reprinted from Baranski et al. (2004) without permission.
Hah.
frustrating the most obvious approach 'just univariate meta-analyze everything!'
I'm curious if you have ideas on how to deal with that.
Maybe grouping the tests into different kinds of tests and fitting a hierarchical model inside those groups? Are there similar kinds of tests?
Modafinil is probably the most popular cognitive enhancer. LessWrong seems pretty interested in it. The incredible Gwern wrote an excellent and extensive article about it.
Of all the stimulants I tried, modafinil is my favorite one. There are more powerful substances like e.g. amphetamine or methylphenidate, but modafinil has much less negative effects on physical as well as mental health and is far less addictive. All things considered, the cost-benefit-ratio of modafinil is unparalleled.
For those reasons I decided to publish my bachelor thesis on the cognitive effects of modafinil in healthy, non-sleep deprived individuals on LessWrong. Forgive me its shortcomings.
Here are some relevant quotes:
Introduction:
Overview of effects in healthy individuals:
Adverse effects:
Abuse potential:
Conclusion:
You can download the whole thing below. (Just skip the sections on substance-dependent individuals and patients with dementia. My professor wanted them.)
Effects of modafinil on cognitive performance in healthy individuals, substance-dependent individuals and patients with dementia