I spent a long time stalling on this post because I was framing the problem as “how to choose a book (or paper. Whatever)?”. The point of my project is to be able to get to correct models even from bad starting places, and part of the reason for that goal is that assessing a work often requires the same skills/knowledge you were hoping to get from said work. You can’t identify a good book in a field until you’ve read several. But improving your starting place does save time, so I should talk about how to choose a starting place.
One difficulty is that this process is heavily adversarial. A lot of people want you to believe a particular thing, and a larger set don’t care what you believe as long as you find your truth via their amazon affiliate link (full disclosure: I use amazon affiliate links on this blog). The latter group fills me with anger and sadness; at least the people trying to convert you believe in something (maybe even the thing they’re trying to convince you of). The link farmers are just polluting the commons.
With those difficulties in mind, here are some heuristics for finding good starting places.
- Search “best book TOPIC” on google
- Most of what you find will be useless listicles. If you want to save time, ignore everything on a dedicated recommendation site that isn’t five books.
- If you want to evaluate a list, look for a list author with deep models on both the problem they are trying to address, and why each book in particular helps educate on that problem. Examples:
- Fivebooks’ Best Books on Learning from the Great Depression
- Fivebooks’ Best Books on Evolution
- A bad list will typically have a topic rather than a question they are trying to answer, and will talk about why books they recommend are generically good, rather than how they address a particular issue. Quoting consumer reviews is an extremely bad sign and I’ve never seen it done without being content farming.
- Jerry Jenkin’s The 12 Best Books on Writing I’ve Ever Read
- Culture Trip’s 7 Must-Read Books Based on India’s History
- Search for your topic on Google Scholar
- Look at highly cited papers. Even if they’re wrong, they’re probably important for understanding what else you read.
- Look at what they cite or are cited by
- Especially keep an eye out for review articles
- Search for web forums on your topic (easy mode: just check reddit). Sometimes these will have intro guides with recommendations, sometimes they will have where-to-start posts, and sometimes you can ask them directly for recommendations. Examples:
- Search Amazon for books on your topic. Check related books as well.
- Ask your followers on social media. Better, announce what you are going to read and wait for people to tell you why you are wrong (appreciate it, Ian). Admittedly there’s a lot of prep work that goes into having friends/a following that makes this work, but it has a lot of other benefits so if it sounds fun to you I do recommend it. Example:
- Dan Luu pointed me to Chicago undergraduate mathematics bibliography when I stated that I’d never seen a list with >5 recommendations that was any good.
- Ask an expert. If you already know an expert, great. If you don’t, this won’t necessarily save you any time, because you have to search for and assess the quality of the expert.
- Follow interesting people on social media and squirrel away their recommendations as they make them, whether they’re relevant to your current projects or not.
I wonder if a good pre-reading strategy is to search for, or ask experts about, the major controversies and challenges/issues related to the topic in question.
Your first step would be to try and understand what those controversies are, and the differences in philosophy or empirical evaluation that generate them. After you've understood what's controversial and why, you'll probably be in a better position to interpret anything you read on the subject.
One way you could potentially further your work on epistemic evaluation is to find or create a taxonomy of sources of epistemic uncertainty. Examples might include:
You can find papers addressing many of these issues with the right Google Scholar search. For example, searching for "controversies economic inequality" turns up a paper titled "Controversies about the Rise of American Inequality: A Survey." And searching for "methodological issues creativity" turns up "Methodological Issues in Measuring Creativity: A Systematic Literature Review."
My guess is that even just a few hours spent working on these meta-issues might pay big dividends in interpreting object-level answers to the research question.
It seems like some questions might seem heavily researched, but are in fact either so hazy that no amount of research will produce clarity, or so huge that even a lot of research is nowhere near enough.
An example of the latter might be “what caused the fall of Rome?”
Ideally, you’d want numerous scholars working on each hypothesis, modeling the complex causal graph, specializing in various levels of detail.
In reality, it sounds like there are some hypotheses that are advanced by just one or a handful of scholars. Without enough eyes on every aspect of the p
... (read more)