Now, what did I miss?
Clearly, as you hinted by noting that studying certain subjects can actually make you dumber, the key problem is how to evaluate the soundness and reliability of the existing literature in a given area as an non-expert. I raised this topic many times on LW, including in a recent top-level article, but there doesn't seem to be much interest in it here -- even though the issue is, in my opinion, of crucial importance, not least because all the arguments about the value and importance of scholarship that you list also hinge on it.
Why do you say there doesn't seem to be much interest? Your post has 48 karma and 244 comments. That doesn't seem like "not much interest" to me.
Now, what did I miss?
One trouble with scholarship is that you risk shifting the discussion from how good your ideas are, to how good your scholarship is. An anecdote backing this opinion:
You wrote:
Suppose you were about to argue, with Jeremy Bentham, that all intentional human action aims at pleasure. Doing some research on the neuroscience of intentional action would help you avoid that mistake. As it turns out, it's just not true that all intentional human action aims at pleasure. Pleasure is only one goal among many.
Why did you insert that clause "with Jeremy Bentham"? It adds nothing to your point, but it does show off your scholarship. And what is wrong with that? Well, (and here is the anecdote), when I read that I immediately thought to myself "I'll bet what Bentham meant by 'pleasure' is not the same as what the neuroscience researchers meant by 'pleasure'". And I became motivated to find out by researching Bentham. Which is completely irrelevant to the point you were trying to make!
In other words, if you are not careful, scholarship can become a shiny distraction.
For my part, I took the "with Jeremy Bentham" clause to be a concise way of saying "Incidentally, this isn't a strawman example intended to artificially support my point; this is a real example of a significant player who made this particular error."
Relatedly, if you had done that research and came back and objected to Luke that his example of Bentham was a bad example, because Bentham is not actually arguing what Luke summarizes him as arguing, I would judge that as doing research that improved the quality of the article.
And relatedly to that, if you were deciding ahead of time whether to do the research, and you estimated that reaching that conclusion was a likely outcome (which it sounds like you did), I would judge that deciding to do it was a sensible decision if you wanted to improve the quality of the article.
Now, whether improving the quality of the article is itself worth doing or not is of course a separate question, but if it isn't, then your comment is itself a shiny distraction (as is mine, and indeed most of my activity on this site, and elsewhere), and we're no longer talking about any special property of scholarship.
After all, sitting around working out solutions from first principles can also be a shiny distraction.
For the record, here are some of my thoughts explaining why I focus on so much scholarship, in no particular order.
First, research is something I'm good at. I've spent a lot of time doing it, and I can do it fairly efficiently. I've developed heuristics for determining very quickly whether something is likely to be useful to my project or not. I know how to figure out which terms are used to describe the concepts of a field that is new to me, and bring myself up to speed very quickly by finding survey articles and review articles and Handbook chapters and so on. Also, I have a pretty strong work ethic and some limited mastery over procrastination - both of which are required for long "literature slogs." So research is a comparative advantage of mine as compared to, say, making cutting-edge advances in AI or decision theory or statistics or neuroscience.
Second, I know that I almost always prefer well-researched writing to poorly-researched writing. I prefer when people name-drop the people or concepts or articles relevant to the topic they are discussing, whether or not they were partially motivated to do so by a desire for prestige. Why? Because then if I don't understand...
What evidence is there that some fields of study make their followers dumber?
That section of the post sounds a bit like a mind-killer shout-out.
If the respectable academic consensus in some field is remote from reality, the prominent authors in it will normally still be strongly selected for intelligence and skills in writing and arguing. As prominent academic authors, they will also be very high-status individuals. It follows that by studying some such field, you are exposing yourself to well-written and masterfully crafted arguments for delusional views espoused by intelligent high-status people. Unless you approach the subject with a hostile stance, it can be very hard to avoid falling for them.
This is especially problematic in fields whose subject matter is ideologically charged. Studying those often means submitting oneself to highly effective ideological propaganda, which can be very hard to resist.
That you carefully avoid naming names is itself a symptom of the problem, as well as contributing to it (I'm not in any way blaming you - you have to protect yourself). You self-censor, along with a lot of other sensible and reasonable people, with the result that the propaganda goes unanswered. Of course a lot of people are not shy about attacking the propaganda, but they either already were or else become low-status, which has the perverse effect of strengthening the propaganda.
It's really very inconvenient for me that you never name names, which leaves the discussion at an abstract and therefore not entirely useful level. Additionally it's not very convincing, because you don't give the material, specific evidence that your claims are true (again, entirely understandable and blameless). It's a bit like discussing the conflict between the creationists and the Darwinists without ever saying that that's what you're talking about, let alone saying which side is right and why. (It is safe to mention the Darwinists versus the creationists, because the right side, the Darwinists, are high status.)
Your phrasing (twice in the essay and now in that comment) is pretty much indistinguishable from proud declaration of ignorance as social signaling. Invoking straw postmodernists is neither big nor clever.
Contrary to many rationalists' views, postmodernism is not composed entirely of bullshit - it is a useful critical method to keep on hand when talking about mushy social and artistic things, like almost all of what humans do that might be called "culture". Humans are incredibly full of shit, and postmodernism and critical theory can be somewhat useful in cutting through it and calling them on it.
However, as the product of humans, it is itself horribly susceptible to bullshit in turn, particularly when overapplied to actual reality. It's also really, really badly lacking in rigor, and pretty much crashes and burns on Vladimir M's tests. So that's a reason not to bother with it unless you're interested in it for its own sake, as I am. I suspect you need to have worked out a usable amount of it yourself to get use out of it.
Nevertheless, it is about something and useful. I'd say that any effective writer of fiction needs a working knowledge of postmodernist techniques, whet...
I'd delight in telling you you're wrong, but you're mostly not.
I would say that I don't think that postmodernism is lacking in rigor. Certainly, having been on both ends of peer review in the humanities, it does not seem to me that the process lets through a lot of flamingly inaccurate crap, beyond the sort of expected problems you get in the margins of well-studied ground. Frankly, in my own research, I'd have an easier time sailing a howler about the history of video games past peer review than I would a howler about the applications of Derrida.
I'm also not sure it does as badly as you say on Vladimir M's heuristics. Looking quickly, for my own field, there's still a ton of low-hanging fruit. Yeah, the major canonical works of literature assigned to undergraduates are pretty well-covered in the literature, but if you're working in popular culture of any era, you have basically no excuse for running out of things to say. The ideology test is a little trickier, since there are areas of literary criticism - feminist, queer, and racial studies, most obviously - that are explicitly ideological. But, of course, we have to be careful with ideology as a warning sign, because arguably at ...
But I think that one can write for a very, very long time about sexual politics in Victorian literature without ever running into a situation where lack of knowledge of science beyond a high school level is going to be a problem. It's certainly difficult to imagine it resulting in nonsense production that goes beyond a stray sentence here or there.
I strongly disagree here. To write meaningfully about sexual politics, you must have a model of sexual and other related aspects of human thought and behavior, and modern science has a whole lot to say about that. (Of course, the relevant science is still very incomplete and far from settled, but that makes it even more important to be knowledgeable about it, in order to separate solid insight from speculation.) If you lack that knowledge, your model is likely to be wrong in at least some ways that could be corrected by familiarizing yourself with the relevant science, and this is likely to show in your writing. Moreover, there is a whole lot of spurious pseudo-insight in this area (Freudianism and its offshoots being the most notorious example), and if you're not familiar with science beyond a high school level, you may well end up swallowing a lot of such nonsense believing it to be solid insight and incorporating it into your work.
I've spent quite a bit of money on scholarship: on gas for trips to a university library so I can download papers from behind the paywall,[...]
This is, of course, obviously and objectively evil (or about as close as it's possible to get). Any LWers care to start a Scholars Bay?
(or at least some private IRC channel where LWers with paywall keys can share them with those without)
Opportunity cost
Scholarship takes time - time that could also be spent on actually doing something instead of just reading about it.
Note that I love scholarship... but I also find it an excellent form of procrastination. You can use the excuse of "I should read up a bit more about this" to put off the hard work of actually shipping something worthwhile. Thus scholarship can become a form of analysis paralysis.
Scholarship is excellent, but it is also expensive. It takes a long time to catch up to the state of the art, even for a narrow subject.
I recently read 90% of the literature on machine ethics, a recent and small field of inquiry, and it took me about 40 hours to find all the literature, acquire it, and read (or skim) through it. Doing the same thing for an older and larger subject will take far more time than that. And of course most of the literature on any subject is not valuable.
Other times, you get lucky. Let's say you want to figure out how to beat procrastination. You could introspect your way to a plausible solution, but you might end up being wrong. So, you check Wikipedia. Not very useful. Next, you search Google Scholar for "procrastination." An article on the first page looks like what you want: an overview of the scientific research on procrastination. It's called "The nature of procrastination: A meta-analytic and theoretical review of quintessential self-regulatory failure," and it's available online! As it turns out, you can do a pretty decent job of catching up on the science of procrastination just by reading one article. (Of course it's not that easy. You should be more thorough, and explore alternate perspectives. Psychology is not settled chemistry.)
And in machine ethics, it turns out that most of what you'd want to know is summarized nicely in a single book: Moral Machines (2009).
But on other topics, you won't be so lucky. Suppose you want to study the neuroscience of how desire works. You check Wikipedia, and it has a section on the psychology and neurology of desire. But it doesn't tell you much. A Google Scholar search is even worse. You check the index of a large neuroscience textbook for "desire," and come up with basically nothing. The Stanford Encyclopedia of Philosophy article on desire is pretty good, but it barely touches on neuroscience. It does point you to two good resources, though: the book Three Faces of Desire, which sounds like it will cover the neuroscience, and the work of neuroscientist Kent Berridge. Now, because I have studied the neuroscience of desire, let me spoil the surprise at this point: this research project is not going to be so easy. You have a very long "literature slog" ahead of you.
So I won't argue that scholarship is always or even usually the instrumentally rational thing to do when you want to make progress on a certain problem. Sometimes the costs of scholarship outweigh the benefits.
But to make that judgment, it will help to know just what those costs and benefits are.
1. Scholarship takes time and effort.
This is the biggie. Scholarship takes time. Especially if you don't have much experience with it already. Resources like Google Scholar make it easier than ever, but scholarship still requires lots of patience and perseverance and procrastination-mastering.
2. Opportunity cost.
The fact that scholarship takes up time means that while you're doing scholarship, you're not doing something else that might be more productive. Scholarship can even serve as a form of procrastination, reading things just because they're on your to-read list so that you can avoid doing something else. [thanks taryneast]
3. Studying some subjects can weaken or corrupt you.
Without (and maybe even with) rationality training, studying certain subjects will make you dumber. To me, postmodernism and even most analytic philosophy look like a good candidates for stupid-making subjects of study. Other candidates include theology, literary theory, and scripture scholarship. These fields can teach bad modes of thinking, false "facts", and even absurdities. Luckily, there are some heuristics you can use to estimate the value of a field.
4. Some things cost money.
I've spent quite a bit of money on scholarship: on gas for trips to a university library so I can download papers from behind the paywall, on hard drives to store tens of thousands of PDFs, on books purchased from Amazon, and so on.
5. Scholarship can be a shiny distraction.
A long list of footnotes and references might be built up to conceal the fact that the ideas in an article or book are of little value. [thanks Perplexed]
1. You'll avoid some mistakes and confusions.
Suppose you were about to argue, with Jeremy Bentham, that all intentional human action aims at pleasure. Doing some research on the neuroscience of intentional action would help you avoid that mistake. As it turns out, it's just not true that all intentional human action aims at pleasure. Pleasure is only one goal among many.
Scholarship can also help you avoid confusions, for example between two kinds of intrinsic value.
2. You'll learn to speak the same language as everyone else, and thus communicate more effectively.
When you read other works on the topic you're discussing, you discover the established terms already used for discussing that topic, and you can begin to speak the same language as everybody else.
3. If your rationality skills are sharp, you'll become generally smarter and wiser.
If you're equipped to recognize magical categories and mysterious answers, consuming a diverse array of fields can give you a broad, integrative kind of knowledge. But be especially wary of subjects that can make you stupid, like postmodern philosophy.
4. You won't waste time re-stating what has already been said elsewhere, better and more knowledgeably than you can.
Many times, I've decided I want to write about X. So then I start researching X to prepare for writing. Then I discover that somebody wrote an article that says everything I wanted to say about X, but they've been studying X for ten years and really know their stuff. Then all I have to do is type two paragraphs about it and link to it on my blog. Hurray!
5. You'll be taken seriously by more people, and have more access to useful experts.
Researching your topic and citing the relevant literature are pre-requisites for some activities like academic publishing, which can get more people to take you seriously because you've put forth the effort to pass a basic test of quality: peer-review. Really smart and accomplished people have too much to read already, and most of them are unlikely to read what you've written if you haven't even bothered to pass peer review.
And, the more smart people take you seriously and read your stuff, the more brain power you can call upon in solving the problems you care about.
6. You'll avoid the tendency to over-trust bearers of good info.
Especially when approaching a subject for the first time, you might read something so bloody intelligent that your mind can't help but cast a halo around its author and accept whatever he or she said. But continuing with your scholarship, and reading lots of people who agree and disagree with that author, can help you see him or her as part of a large enterprise that has been struggling on certain problems for a long time, and may expose you to data that disproves claims made by the original author that impressed you.
Nobody on Less Wrong should be fooled by the fact that I listed more benefits than costs for scholarship. That doesn't mean scholarship is always a good idea. Often, it's not a good idea.
But having a list of costs and benefits can help you decide whether scholarship is worthwhile for a particular project - or, how much scholarship is worthwhile.
Now, what did I miss?