Desrtopa comments on Some Heuristics for Evaluating the Soundness of the Academic Mainstream in Unfamiliar Fields - Less Wrong

73 Post author: Vladimir_M 15 February 2011 09:17AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (272)

Sort By: Popular

You are viewing a single comment's thread.

Comment author: Desrtopa 19 February 2011 05:12:14AM *  16 points [-]

As the first heuristic, we should ask if there is a lot of low-hanging fruit available in the given area, in the sense of research goals that are both interesting and doable. If yes, this means that there are clear paths to quality work open for reasonably smart people with an adequate level of knowledge and resources, which makes it unnecessary to invent clever-looking nonsense instead. In this situation, smart and capable people can just state a sound and honest plan of work on their grant applications and proceed with it.

In contrast, if a research area has reached a dead end and further progress is impossible except perhaps if some extraordinary path-breaking genius shows the way, or in an area that has never even had a viable and sound approach to begin with, it’s unrealistic to expect that members of the academic establishment will openly admit this situation and decide it’s time for a career change. What will likely happen instead is that they’ll continue producing output that will have all the superficial trappings of science and sound scholarship, but will in fact be increasingly pointless and detached from reality.

This sounds like a useful heuristic, but I think there's another one almost directly opposed to it which is worth keeping in mind. In some branches of psychology, for instance, there is so much low hanging fruit that you'd think that researchers would never have a shortage of work. But instead, entire schools of psychology have persisted based on conclusions drawn from single experiments which were never followed up with the appropriate further research to narrow down the proper interpretation. I've been told that sociology and anthropology suffer similar issues.

If a field (or sub-field) doesn't exhibit enough interest in pursuing low hanging fruit, I think that's a good sign that there's a high ratio of ideological rationalization to solid research.