Your point that most self-help books are not peer-review backed is very true, and something more people should be aware of. OTOH, I did trials with a number of self-help books, and how useful they were to me and my friends did not correlate with scientific rigor. https://acesounderglass.com/2018/04/14/self-help-epistemic-spot-check-results/ . Given that, I think it makes sense to put more emphasis on individual testability.
This is a fair point, and I find it interesting that you did this and came to the conclusion "nothing short of trying it seems to have any predictive ability of whether or not it is helpful". When you hand somebody a self-helf book, it is hard to say what the exact treatment should be that will be tested. If the treatment is just handing out the book, then it may be that bad writing keeps the treatment effects low. If the treatment is applying all techniques, then the test for some of the books will be very expensive, and the author can make up excuses why something does not work.
A topic I considered to cover but postponed is cognitive dissonance; if people have invested time into following a technique, they want to find it useful afterwards. Basically, people then apply the same biases that make them prey to cold reading and other con-artist techniques: They attribute positive developments to having read the book, and ignore counter-evidence. Some of the self-help authors have a large cult around them, probably starting with Napoleon Hill. Cognitive dissonance also applies to self-help communities without a guru at the center, because fans mutually reinforce their beliefs. Eye training may be an example.
But of course not everything under the book-shelve label "self-help" is like that. The following things should make you very suspicious:
- The author writes in a very obscure style, and it is hard to grasp what he is talking about.
- The author claims to have discovered a magical secret, as opposed to either scientifically-based advice or commonsense observations.
- You decide to apply the techniques, do it for a while, but nothing happens (of which it is not more likely that it has nothing to do with the book).
- The techniques are basically not applicable.
- The techniques depend strongly on belief (If you don't believe it, it doesn't work.)
The epistemic problem here is that many of these things can "work" on some level, like a mental placebo. But they work because e.g. you have decided to change your life, become more confident etc. and not because of some magic the author describes. Maybe you just feel better because you took time to read a book instead of just sitting there worrying. Maybe you imagined yourself in a better state and that made you feel better. I find it hard even to define a placebo for that. In all these cases, the willingness to start is more important than the content of the book, which of course poses a testability problem for the content. (I believe this is also a problem for the testability of recognized psychotherapies.)
Many people want to change themselves. They want to become happier, smarter, more successful, rich, or at least less disoriented. There are incredibly many books telling people how they can achieve this, along with seminars, youtube videos and email instructions. And the large majority of this self-improvement literature is nonsense.
There is of course a wide range of what the authors promise. But the typical "If you follow this author's system, then you will get rich / become world ruler / can do whatever you want" has some features worth noting.
The literature sells well because people are unhappy with their life, their abilities, their habits and *want* to have other lifes, other abilities, other habits, and there is only a limited number of ways to cope with this, the most apparent may be: Do nothing and be unhappy, find ways to attribute responsibility to external circumstances, find new perspectives or other ways to be happy without changing the things that you think make you unhappy - and finally, listen to people who tell you that they have a clue. Similarly, if you are not unhappy but just very ambitious and, in your own eyes, not very successful, you can try to be ok with that tension, or find authors promising you change. In any case, you have made the first step of blaming yourself, but in a way that you think you can change - as opposed to blaming your genes alone, or society. Thus, firstly, self-improvement books have to overemphasize what you can change over what you can't - it is their selling point. So, first advice: Be skeptical of authors who claim that everything is possible. If something sounds too good to be true, it probably is.
Secondly, the advice is rarely evidence-based, and the greater the promises, the less valid evidence. This, of course, should make a reader suspicious, but only if he has some understanding of scientific methods. Because many of the authors know this, there seems to be a tendency towards faked scientifc credibility in parts of the literature, namely those parts that want to seem not only credible, but also serious. These authors do not cite sources, but vaguely refer to studies conducted at some university, like the famous 1953 Harvard study or the 1979 Yale study on the effects of writing down goals on life success (Mike Morrison has done the research - these studies have not been conducted, and authors copied the story from each other.). So, second advice: Be suspicious if the author claims that his advice is backed up by scientific evidence and then he does not really cite studies that you can actually look up.
Thirdly, the authors often seem themselves very successful in what they promise to teach. And this they should be - the natural question otherwise would be: If you're so smart, then why aren't you rich? But the success of authors is not sufficient prove for the system to work. If you buy a get-happy-and-successful book from the bestsellers' shelve in a large bookstore, the authors *are* very likely to be successful in happy and successful. Otherwise, you would probably not have found their book. An author who has made it to be something between a coach and a guru may possibly even attribute this to his own system. But suppose that you would let 1000 people write self-help books, all results are useless, but *randomly* 5 of them are successful because there is demand for five books. Then this would not tell you anything about the quality of their insights -- assuming the opposite leads to survivorship bias. In reality, people with better "systems" may have better chances of becoming successful in life, but so may people who are just more overconfident, who are charismatic, or who write well. (Nothing of this helps the reader.) What would be necessary is a randomized controlled study of people following a certain system and see how many of them were successful. Thus, third advice: As long as this does not exist, do not put too much weight on the author's own story.
Fourthly, for really ambitious self-optimization claims, unscrupulousness, fraud, conman writing additionally comes to the mix. ("Additionally" because for these authors it is true as well that you more often see the successful writers' books.) It is not always clear whether the more esoteric writers talking of psychic energies and wishes directed to the universe in specific ways are conscious fraudsters or fooled by their own random success. However, there were famous self-improvement authors who were most certainly knowing what they were doing in this respect (Budge Burgess provides a very interesting story about Napoleon Hill) and still had enormous success. The problem with conmen is that they are good at what they do, and they will rarely tell you that the recipe for success is to manipulate readers (though there is the old joke of the book 'how to get rich' that only contains the advice 'write a book like this'). So, fourth advice, be extra skeptical if someone promises extraordinary things -- and ask yourself something like: If that were true then why isn't this guy president, or Apple CEO, or something like that?
Fifth point. In the last years, but maybe already earlier, I have seen books that study certain features of the behavior of very successful people -- tings that successful people do, how artists plan their day, what genius space warlords eat for breakfast etc. (The already-mentioned Napoleon Hill claimed to have done something similar.) This is Cargo Cult par excellence: Imitating something without really knowing whether it has relevance. Imitating a genius probably won't make you one. (As Tim Harford comments on a distraction-free writing tool called the Hemingwrite, "It is probably not true that Facebook is all that stands between you and literary greatness.") Similar to the successful authors, there are no randomized controlled experiments about who becomes a genius if she applies this or that behavior, as far as I know. Even then, one would have to keep in mind that treatment effects may interact: Give n persons with IQ>160 some goal-setting and structure and *maybe* they work wonders. The normal person's output may not change much, as workplaces and institutions are already structured around their abilities and habits. Thus, even if you find fascinating whether Einstein wore socks (something Christian Ankowitsch has written a book about, but it seems more for entertaining than for extraordinary promises), do not expect imitating him to have much of an effect (fifth advice).
Sixth point and advice. Keep in mind that many self-help books have an inbuilt mix of blame allocation and non-falsifiability. If you don't succeed, you didn't follow the advice hard enough, maybe not believe in it enough etc. All this means that you are probably still unhappy but now you can additionally blame yourself for another failure. Moreover, there are tendencies that people want to believe in the promises of such books - in particular, if they paid a lot in terms of money or time - to reduce cognitive dissonance. Be aware of this behavior.
So what to take of this? Are there actually useful books out there, and postitive rules to find them and use them?
Firstly, trust your commonsense. If you read claims that let you say "Now this is something I should have done all my life, why did noone tell me that?", then just do it! (Classical candidate: Dale Carnegie, "How to win friends and influence people") If you think it is fraud, it probably is (Possibly it is not, but time is finite! But also the more basic self-organization and time-management books are in this category)
Secondly, there are some science-based strategies for changing your life, and there are authors getting them to a readable book format, e.g. Wiseman's book "59 seconds". Usually, you will find that the advice of these authors is relatively practical, while their claims are comparably modest.
Thirdly, there are many self-improvement techniques of which you can at least see that they work for *someone*. For instance, if you want to grow muscles, you can go the gym and actually see people who have grown muscles. There may again be survivor's bias at work, but the next step is to read actual, serious advice here. Another case is mnemonics. You can read books by memory champions who explain what they do, and you can see them perform in contests. This usually is a good sign.
Fourthly, it is a good sign if the advice book has low-hanging fruits advice: Things you can just try and then decice whether it helps. This is the case with the mnemonics books, but also for how-to-meditate books (e.g., "10% Happier" by Dan Harris).
Finally, you probably will not acquire superpowers. Be realistic. The scientific literature on becoming a super performer (you need deliberate practice for 10,000 hours to be in the world top class) is much more demanding than the one on getting a bit happier (even therapies are not 10,000 hours). But you can also easily fulfill more realistic claims. Learning to play 'Lady in Black' on the guitar is much easier than becoming rich and famous. Maybe that is what you really wanted all along?
Given all the advice I have put in this, you may think: If you're so smart, then why aren't you rich? I'll leave it to you how to cope with this paradox.