So, their methodology is, as far as I can tell, described by these parts:
The aim of our meta-analysis was to assess the effect a mindfulness meditation intervention on health status measures. We considered the concept of health to include both physical and mental health. All outcome measures were either subsumed under "physical health", "mental health" or were excluded from the analysis. We only included data from standardized and validated scales with established internal consistency (e.g., the Global Severity Inventory of Symptom Check List-R, Hospital Anxiety and Depression Scale, Beck Depression Inventory, Profile of Mood States, McGill-Melzack Pain-Rating Scale, Short Form 36 Health Survey, and Medical Symptom Checklist; a full list is available upon request). Also a conservative procedure was chosen to exclude relatively ambiguous or unconventional measures, e.g., spiritual experience, empathy, neuropsychological performance, quality of social support, and egocentrism.
"Mental health" constructs comprised scales such as psychological wellbeing and symptomatology, depression, anxiety, sleep, psychological components of quality of life, or affective perception of pain. "Physical health" constructs were medical symptoms, physical pain, physical impairment, and physical component of quality of life questionnaires.
...We first integrated all effect sizes within a single study by the calculation of means into two effect sizes, one for mental and one for physical health. If the sample size varied between scales of one study, we weighted them for N. Effect sizes obtained in this manner were aggregated across studies by the computation of a weighted mean, where the inverse of the estimated standard deviation for each investigation served as a weight [8].
So, they just split the effect sizes, and do an average of the 2 sets. Nothing more.
I dunno. They don't give any references to papers or textbooks on meta-analysis to justify this procedure. It doesn't sound very kosher to me.
From a statistical point of view, I wouldn't expect this to work very well. I would expect a lot of heterogeneity and a very weak signal. However, they report very strong results with low heterogeneity (which I find pretty surprising). I don't see any obvious way in which this would be "cheating".
Are you worried about something else specific?
Modafinil is probably the most popular cognitive enhancer. LessWrong seems pretty interested in it. The incredible Gwern wrote an excellent and extensive article about it.
Of all the stimulants I tried, modafinil is my favorite one. There are more powerful substances like e.g. amphetamine or methylphenidate, but modafinil has much less negative effects on physical as well as mental health and is far less addictive. All things considered, the cost-benefit-ratio of modafinil is unparalleled.
For those reasons I decided to publish my bachelor thesis on the cognitive effects of modafinil in healthy, non-sleep deprived individuals on LessWrong. Forgive me its shortcomings.
Here are some relevant quotes:
Introduction:
Overview of effects in healthy individuals:
Adverse effects:
Abuse potential:
Conclusion:
You can download the whole thing below. (Just skip the sections on substance-dependent individuals and patients with dementia. My professor wanted them.)
Effects of modafinil on cognitive performance in healthy individuals, substance-dependent individuals and patients with dementia