Short response: Check out the Cochrane Library on mental health. (Browse by Topics in the left-hand side, Expand, then click on Mental Health - as of just now there are 406 entries.)
Evaluating healthcare interventions is hard. The gold standard is a randomised controlled trial (RCT), published in a peer reviewed journal. But there are all sorts of problems with single trials, some of which you allude to here. It's a really great idea to do a systematic review of all published trials and combine the good ones to get the best evidence available.
Doing this well is really hard - you need specialist expertise in the specific area to correctly interpret the primary literature (the RCTs), and specialist skills in systematic reviewing (as with RCTs themselves, there are many obvious and subtle issues about how to do them well). And it takes ages.
Luckily, there's an international collaboration of people, called the Cochrane Collaboration who get together to do this sort of thing, and have been beavering away for 20 years.
Unless you have significant resources, you are unlikely to do better on any topic than the latest available Cochrane Review. And if you do have significant resources, you're likely to do well to start with it.
When a health issue pops up for me or someone I care about, I jump straight for the Cochrane review (and also any relevant guidelines and protocols, but that's a tier down the evidence quality pyramid), and it's like I'm getting a well thought-through briefing from the world's experts on what we currently know about what works and what doesn't.
I love it.
As a postscript, there is a whole field of healthcare informatics that looks at how to find good academic papers on a particular issue - I once ran a whole course on the topic (and related ones). The shortcut answer is 'use Cochrane'; the long spadework answer is 'search Medline'.
Good luck.
I wasn't familiar with Cochrane; that looks like an excellent resource. Unfortunately, it looks like a lot of summaries haven't been updated in a decade - is this something to be worried about, and if so, is there another resource someone can recommend other than simply reading PubMed and doing your own meta-analysis?
This is a thread where people can ask questions that they would ordinarily feel embarrassed for not knowing the answer to. The previous "stupid" questions thread is at almost 500 questions in about a month, so I think it's time for a new one.
Also, I have a new "stupid" question.