tangent: System 1 seems to control how "profound", and thus likely-to-apply-in-the-future, any given concept feels. Venkatesh Rao has written a piece on this I can't find right now, but the gist was that we glom onto concepts that allow more efficient mental organization. For example discovering that two phenomenon we thought were separate are actually sub-cases of some more basic phenomenon. An important point is that we do this speciously, as our pattern recognition is overactive (it is worth false alarms when checking for leopards). This predicts wide ranging failures such as religion, policy wonkery, conspiracy theories, etc.
Anyway, the point is that my process for finding, evaluating, and adding such concepts to my permanent repository of cognitive tools is not well defined and this bothers me. I've tried explicitly adding concepts to my permanent toolbox without being positive they will be helpful, for example when I used Anki (spaced repetition software) to help remember biases and fallacies. I found it hard to stick with this even though it did in fact seem to help me notice when making specific errors more. So I guess what I'm basically asking is why aren't we spending a lot more time improving the checklist of rationality habits, especially via empiricism.
For example discovering that two phenomenon we thought were separate are actually sub-cases of some more basic phenomenon.
I like to do this a lot in mathematics, but fortunately mathematical language is both rich and rigorous enough that I can avoid false alarms in that context (category theory in particular is full of examples of phenomena that look separate but can rigorously be shown to be subcases of a more basic phenomenon).
I think of mathematics as being a conspiracy theorist's fantasy land: it works nearly the way a conspiracy theorist thinks re...
See also: Boring Advice Repository, Solved Problems Repository, Grad Student Advice Repository
I often find that my understanding of the world is strongly informed by a few key concepts. For example, I've repeatedly found the concept of opportunity cost to be a useful frame. My previous post on privileging the question is in some sense about the opportunity cost of paying attention to certain kinds of questions (namely that you don't get to use that attention on other kinds of questions). Efficient charity can also be thought of in terms of the opportunity cost of donating inefficiently to charity. I've also found the concept of incentive structure very useful for thinking about the behavior of groups of people in aggregate (see perverse incentive).
I'd like people to use this thread to post examples of concepts they've found particularly useful for understanding the world. I'm personally more interested in concepts that don't come from the Sequences, but comments describing a concept from the Sequences and explaining why you've found it useful may help people new to the Sequences. ("Useful" should be interpreted broadly: a concept specific to a particular field might be useful more generally as a metaphor.)