Great point! One of the problems here is that people think that just knowing about something is going to give them the power but this is not the case. Rationality is a skillset like bicycle riding or playing chess and the only way to get good at it is by practicing a lot. You can read lots of books of chess and get great insights but when it comes down to actually playing at the board what matters is what you have internalized through practice.
Even worse, unlike your examples, rationality isn't a single, focussed "skillset", but a broad collection of barely related skills. Learning to avoid base rate neglect helps little if at all with avoiding honoring sunk costs which helps little with avoiding the narrative fallacy. You need to tackle them almost independently. That is one reason why I tend to emphasize the need to stop and think, when you can. Even if you have not mastered the particular fallacy that may be about to trip you up, you are more likely to notice a potential problem if you get in the habit of thinking through what you are doing.
LW doesn't seem to have a discussion of the article Epiphany Addiction, by Chris at succeedsocially. First paragraph:
I like that article because it describes a dangerous failure mode of smart people. One example was the self-help blog of Phillip Eby (pjeby), where each new post seemed to bring new amazing insights, and after a while you became jaded. An even better, though controversial, example could be Eliezer's Sequences, if you view them as a series of epiphanies about AI research that didn't lead to much tangible progress. (Please don't make that statement the sole focus of discussion!)
The underlying problem seems to be that people get a rush of power from neat-sounding realizations, and mistake that feeling for actual power. I don't know any good remedy for that, but being aware of the problem could help.