= 27d567bc2d48d9f40e9ccc20ea370b15)
evgenit comments on The Stable State is Broken - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (43)
Reminds me of Scott Aaronson's Malthusianisms. Is this the article you couldn't find?
Also, I am not sure your example of science is correct: After all, plenty of very famous journals do publish retractions, and some that do not are (rightly) laughed at (parapsychology journals for example).
That's it exactly! Thanks much. (It's a little disappointing how close my article comes to his, even stylistically. This was completely unintentional; I hadn't read his since it was published.)
Re: journals, I've edited it to also mention reproductions (which are accepted much less frequently). I should have mentioned reproductions in the first place, although what I was thinking of was more failures to reproduce results (as in the parapsychology example).
The idea seems to be in essence a restatement or reapplication of Campbell's Law: A system's metrics and its goals are not equivalent, so successful behavior tends to become deranged. Though they are not quite the same idea.
Not to say that expounding upon the idea is a bad thing. Your examples are quite elegant.
Campbell's Law posits that a metric is used because it has historically correlated with a difficult-to-measure desirable property; but that it becomes deranged only when it is used to make decisions that the measured people care about.
Historically, better-educated students do well on standardized tests, when those tests don't matter. But once you enact a test to discriminate amongst students for purposes those students care about (like getting into prestigious colleges), your measurement of academic achievement will be confounded by your measurement of test-taking skills.
Certainly. I think the same principle applies in many of the listed cases, though. Scientific publications in particular likely developed their current standards at least partially because in the past they filtered for genuinely revolutionary results.
One might as easily say that Campbell's Law is a sub-principle of the observed phenomenon.
Replications make much more sense as an example. You could also add the file-drawer problem in research. Why do we not see studies that do not find anything? Because there is no prestige in publishing them. (Some journals do try to correct for this, but they have to explicitly do that)
Your article and Aaronson's are similar in tone and superficially similar in content, but yours embodies a more specific and more interesting (to me) central idea. To wit, Malthusianisms is basically about Umeshisms, which is really a restatement of the Pareto Principle. Whereas this article is about the Hansonesque idea that success in a domain is not about proficiency in the domain.