Mass_Driver comments on The Neglected Virtue of Scholarship - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (153)
In general I'm very sympathetic to this point of view, and there are some good examples in your post.
One bad example, in my opinion, is Eliezer's recent procrastination post vs. the survey of "scientific research on procrastination." I read the chapter, and it appears to mostly cite studies that involved casual surveys, subjective description, and fuzzy labeling. Although there are many valid scientific endeavors that involve nothing but categorization (it is interesting to know how many species of tree frog there are and what they look and sound like even if we do not make any predictions beyond what is observed), categorization should at least be rigorous enough that we can specify what we expect to see with a modicum of precision.
When a biologist says that frogus neonblueicus has neon blue spots and chirps at 500 Hz, she will give you enough information that you can go to Costa Rica and check for yourself whether you have found one of the rare neonblueicus specimens. Although there will be some controversies around the edges, your identification of any particular frog will not correlate with your political biases or personal problems, and repeated observation of the same frog population by a few different researchers will tend to decrease error.
When a psychologist says that procrastinators can be divided into "relaxed" types and "tense-afraid" types, the "science" being done is not merely descriptive, but also horrifyingly vague. What does it mean for a human to be "tense-afraid" when "procrastinating"? The three paragraphs or so of context on the topic give you enough of an idea of what the researcher is saying to conjure up a mental image, but not nearly enough to carve thing-space at the joints.
In my experience, this is a very serious problem in social and human sciences -- there are whole subfields where the authors do not know how little they know, and proceed to wax eloquently about all of the empty concepts they have coined. There are other subfields where the researchers suspect that they might not have done very good research, and they cover their tracks with advanced statistics and jargon. After you dig through a few of these booby-trapped caves of wonder, you start to lose, if not respect for scholarship, at least some of the urge to do the moderately hard work of digesting literature reviews yourself on a regular basis. It is dangerous to assume that casually studying the leading textbook in a soft field will usually make you smarter.
I very much agree with your final sentence.
Do you think Eliezer's post is more precise and useful than the controlled experiments published in peer-reviewed journals described in the book I linked to? I find that most writing on psychology is necessarily pretty soft, because the the phenomena it is trying to describe are vastly more complicated than those of the hard sciences.
Now, that link is a must-read. I got through the whole first chapter before I could look away, and I'll be going back for the rest.
I have nothing against psychology or psychologists or social science in general -- AP Psych was my second favorite class in high school, my mom has a master's degree in it, my bachelor's degree is in political science, etc. It's noble, hard work, and we even have a little bit of knowledge to show for it.
As for the "controlled experiments" described in the book you linked to, I'm afraid I missed them, for which I apologize. I only saw descriptive papers. Maybe a page reference when you get a chance? Or just link directly to one or two of the studies or the abstracts?
Oops, you're right that my link does not mention controlled experiments. A few controlled experiments are instead mentioned in other sections of the book on techniques applicable to a greater variety of behavior change goals.
Unfortunately, the author of Psychological Self-Help died last year, and his book has not been updated much in the past decade. Of course, more work on procrastination has been done in recent years, though I'm not sure if it is collected nicely anywhere.