From a statistical point of view, I wouldn't expect this to work very well. I would expect a lot of heterogeneity and a very weak signal. However, they report very strong results with low heterogeneity (which I find pretty surprising). I don't see any obvious way in which this would be "cheating".
Are you worried about something else specific?
I don't see any obvious way in which this would be "cheating".
Oh, that's easy: publication bias. If the original studies report only the measures which reached a cutoff, and the null is always true, then since their measures will generally all be on the same subjects/with the same n, their effect sizes will have to be fairly similar* and I'd expect the i^2 to be low even as the results are meaningless.
* since p is just a function of sample size & effect size, and the p threshold is fixed by convention at 0.05, and sample size n is pretty m...
Modafinil is probably the most popular cognitive enhancer. LessWrong seems pretty interested in it. The incredible Gwern wrote an excellent and extensive article about it.
Of all the stimulants I tried, modafinil is my favorite one. There are more powerful substances like e.g. amphetamine or methylphenidate, but modafinil has much less negative effects on physical as well as mental health and is far less addictive. All things considered, the cost-benefit-ratio of modafinil is unparalleled.
For those reasons I decided to publish my bachelor thesis on the cognitive effects of modafinil in healthy, non-sleep deprived individuals on LessWrong. Forgive me its shortcomings.
Here are some relevant quotes:
Introduction:
Overview of effects in healthy individuals:
Adverse effects:
Abuse potential:
Conclusion:
You can download the whole thing below. (Just skip the sections on substance-dependent individuals and patients with dementia. My professor wanted them.)
Effects of modafinil on cognitive performance in healthy individuals, substance-dependent individuals and patients with dementia