This recent blog post strikes me as an interesting instance of a common phenomenon. The phenomenon looks like the following; an intellectual, working within the assumption that the world is not mad, (an assumption not generally found outside of the Anglo-American Enlightenment intellectual tradition) notices that some feature of the world would only make sense if the world was mad. This intellectual responds by denouncing as silly one of the few features of this vale of tears to be, while not intelligently designed, at least structured by generalized evolution rather than by entropy. The key line in the post is
"Conversely in all those disciplines where we have reliable quantatative measurements of progress (with the obvious exception of history) returning to the original works of past great thinkers is decidedly unhelpful."
I agree with the above statement, and find that the post makes a compelling argument for it. My only caveat is that we essentially never have quantitative measures of progress. Even in physics, when one regards not the theory but the technique of actually doing physics, tools and modes of thought rise and fall for reasons of fashion, and once widespread techniques that remain useful fall into disuse.
Other important techniques, like the ones used to invent calculus in the first place, are never adequately articulated by those who use them and thus never come into general use. One might argue that Newton didn't use any technique to invent calculus, just a very high IQ or some other unusual set of biological traits. This, however, doesn't explain why a couple of people invented calculus at about the same time and place, especially given the low population of that time and place compared to the population of China over the many centuries when China was much more civilized than Europe.
It seems likely to me that in cases like the invention of calculus, looking at the use of such techniques can contribute to their development in at least crude form. By analogy, even the best descriptions of how to do martial arts are inadequate to provide expertise without practice, but experience watching experts fight is a valuable complement to training by the relatively inept. If one wants to know the Standard Model, sure, study it directly, but if you want to actually understand how to do the sorts of things that Newton did, you would be advised to read him, Feynman and yes, Plato too, as Plato also did things which contributed greatly to the development of thought.
Anyone who has ever had a serious intellectual following is worth some attention. Repeating errors is the default, so its valuable to look at ideas that were once taken seriously but are now recognized as errors. This is basically the converse of studying past thinkers to understand their techniques.
Outside of physics, the evidence for progress is far weaker. Many current economists think that today we need to turn back to Keynes to find the tools that he developed but which were later abandoned or simply never caught on. A careful reading of Adam Smith and of Ben Franklin reveals them to use tools which did catch on centuries after he published, such as economic models of population growth which would have predicted the "demographic transition" which surprised almost all demographers just recently. Likewise, much in Darwin is part of contemporary evolutionary theory but was virtually unknown by evolutionary biologists half a century ago.
As a practical matter a psychologist who knew the work of William James as well as that of B.F. Skinner or an economist who knows Hayek and Smith as well as Samuelson or Keynes is always more impressive than one who knows only the 'modern' field as 'modern' was understood by the previous generation. Naive induction strongly suggests that like all previous generations of social scientists, today's social scientists who specialize in contemporary theories will be judged by the next generation, who will have an even more modern theory, to be inferior to their more eclectic peers. Ultimately one has to look at the empirical question of the relative per-capita intellectual impressiveness of people who study only condensations and people who study original works. To me, the latter looks much much greater in most fields, OK, in every field that I can quickly think of except for astronomy.
To the eclectic scholar of scholarly madness, progress is real. This decade's sludge contains a few gems that weren't present in the sludge of any previous decade. To the person who assumes that fields like economics or psychology effectively condense the findings of previous generations as background assumptions to today's work, however, progress means replacing one pile of sludge with another fashionable sludge-pile of similar quality. And to those few whom the stars bless with the coworkers of those who study stars? Well I have only looked at astronomy as through a telescope. I haven't seen the details on the ground. That said, for them maybe, just maybe, I can endorse the initial link. But then again, who reads old books of astronomy?
As for Newton's exact mental processes, they are lost to history, and we are not going to get very specific theories about them. Newton can only give us an outside view of the circumstances of discovery. His most important finds were made alone in his private home and outside of academic institutions. Eliezer left school early himself. Perhaps a common thread?
Teachers select strongly for IQ among students when they have power to choose their students. This might be a more powerful aggregator of high-IQ individuals than transmission from parents to children. It might be the case that teachers don't transmit any special powers to their students, but just like to affiliate with other high-IQ individuals, who then go on to do impressive things.
At a certain level of IQ (that of Yudkowsky, Newton) pedagogy becomes irrelevant and a child will teach itself, given the necessary resources. At this point, teachers are more likely to take credit for natural talent while doing nothing to aid it than they are to "transmit intellectual power."
If academic lineages are due to an ability that teachers have to identify talent, this ability is extremely common and predicts achievement FAR better than IQ tests can. I am struck by the degree to which the financial world fails to identify talent with anything like similar reliability.
Also, the above theory is inconsistent with the extreme intellectual accomplishments of East Asians, and previously Jews, within European culture and failure of those same groups to produce similar intellectual accomplishments prior to such cultural admixture.