Imagine reading about the following result buried in a prestigious journal:

We administered [Drug X] to 10,000 patients 80+ years of age selected to be a statistical representation of the populace. None had exhibited any prior medical history to suggest unusual conditions, outside of the normal range of issues collected over a lifetime. 1/3 of the patients were selected as a control group, and the others were entered into a longitudinal study of [Drug X] in which they were given varying doses over a 30 year timespan. [Please read charitably and flesh this out to be a good, well run longitudinal study by your personal standards. The important thing is the number of patients involved.] 

Of the patients administered [drugx] 1x/month for 10 years, we found that there was an increase of average lifespan by 1 year compared to normal actuarial tables. We are unsure of the cause of this. We also had one patient who has yet to die after 30 years and shows no signs of aging. Our drug has effectively demonstrated its properties as a medication designed to reduce cholesterol and will proceed to be approved for normal prescription.

Now, personally, reading this I would be completely uninterested in the normal result and fascinated by the one, crazy, outlier. Living to the age of 110 is abnormal enough that within 6,666 people selected as a statistical representation of the population, it is extremely unlikely that anyone would live that long, much less continue performing at the apparent health of an 80 year old.

How small would the sample size have to be before you would consider trying the drug yourself, just to see if you, too, lived forever as long as you took it? What adverse effects and hassles would you go through to try it? Would these factors interact to influence your decision (Mild headaches and a pill 4x/day in exchange for maybe apparent eternal life? Sign me up!)

This example is an oversimplification to make a point- often in clinical trials there are odd outliers in the results. Patients who went into full remission, or had a full recovery, or were cured of schizophrenia completely.

In the example above, if the sample size had been 10 people, 9 of whom had no adverse effects and one who lived forever, I would take it. I have been known to try nootropics with little or no proven effect, because there are outliers in their samples who have claimed tremendously helpful effects and few people with adverse effects, and i want to see if I get lucky. I think that if even the right placebo could cause changes which improve my effectiveness, it would be worth a shot.

As far as I know, psychiatrists cannot reliably predict that a given drug will improve a patient's long-term diagnosis, and psychiatrists/psychologists cannot even reliably agree on what condition a patient is manifesting. Mental disorders appear to resist diagnosis and solution, unlike, say, a broken leg or a sucking chest wound. I have learned that Cognitive Behavioral Therapy (CBT) has consistent results against a number of disorders, so I have endeavored to learn and apply CBT to my own life without a psychologist or psychiatrist. It has proven extremely effective and worthwhile.

Here is the topic for discussion:  should we trust psychiatric analysis using frequentist statistics and ignore the outliers, or should we individually analyze psychiatric studies to see if they contain outliers who show symptoms which we personally desire? Should we act differently when seeking nootropics to improve performance than we do when seeking medication for crippling OCD? Should we trust our psychiatrists, who are probably not very statistically savvy and probably don't read the cases of the outliers?

Where are the holes in my logic, which suggests that psychiatrists who think like medical doctors/general practitioners have a completely incorrect perspective (the law of averages) for finding and testing potential solutions for the extremely personalized medicinal field of psychotherapy/psychiatry (in which everyone is, actually, an extremely unique snowflake.).

This is more of a thought-provoking prompt than a well-researched post, so please excuse any apparent assertions in the above, all of which is provided for the sake of argument and arises from anecdata.