I've seen one paper arguing against Planck's claim:
Unquestionably, there are scientists in every generation who tenaciously cling to knowledge they learned in their youth, and who refuse to consider new theories that challenge fundamental beliefs. The life-long resistance of Joseph Priestley to oxygen theory, Louis Agassiz to evolutionary theory, and Harold Jeffreys to continental drift are among the notable cases. It is virtually a truism that the last adherents to a fading scientific tradition will be elderly scientists. Yet documented episodes where resistance of isolated individuals crystallizes into generational disputes, or where an ageing scientific elite actually delays community-wide adoption of a new idea, are exceedingly rare. A review of the historical record suggests, on the contrary, that the period of active dissemination and adoption of scientific innovations - even those of revolutionary proportion - is typically shorter than that required for one generation of scientists to replace another. [...]
Curiously, the episode which prompted Planck's observation - the 'controversy' surrounding his youthful reformulation of the second law of thermodynamics - seems a poor illustration of the 'fact' Planck claims to have learned. According to Planck's own sketchy chronology (he provides few dates), not much more than ten years seems to have elapsed between his first unsuccessful attempts at gaining recognition, and the 'universal acceptance' of his dissertation thesis on the irreversible process of heat conduction. Nor does it appear that age was an important factor influencing adoption of the theory. Wilhelm Ostwald, one of the leaders of the opposition 'Energetics' school prominently mentioned by Planck, was only five years older than Planck, whereas Ludwig Boltzmann, whose theoretical work on entropy, in no small measure (as Planck grudgingly concedes) helped bring the scientific community around to Planck's view, was fourteen years Planck's senior.
Some quantitative data permit more systematic examination of age differences in receptivity, for both Lavoisier's and Darwin's landmark contributions. In a study of the Chemical Revolution, McCann reports a negative correlation between author's age and the use of the oxygen paradigm in scientific papers written between 1760 and 1795. On closer inspection of the data, he finds that the earliest group of converts to the oxygen paradigm (between 1772 and 1777) were middle-aged men with close ties to Lavoisier; the inverse age effect became manifest only after 1785, during the ten-year period of 'major conversion and consolidation'. McCann also contends that the age structure of the British community during the latter half of the eighteenth century impeded acceptance of the new theory. In contrast to the declining age of French scientists during this period, the increasing average age of British scientists held back the pace of acceptance of oxygen theory among British scientists of all age strata.
As for evolutionary theory, Hull and his colleagues find weak support for 'Planck's Principle' among nineteenth-century British scientists. The small minority of scientists who held out against the theory after 1869 were, on average, almost ten years older than earlier adopters. Age in 1859 (the year the Origin of Species was published) was unrelated, however, to speed of acceptance for the great majority of those converting to evolutionary theory by 1869. [...]
... we can distinguish high- risk and low-risk contexts for theory choices of individual scientists. A high-risk context is one in which there is substantial resistance to the new theory. Prevailing scientific opinion views it as controversial, a heretical assault on existing knowledge, or even being beyond the pale of serious scientific discourse. Adoption of a new theory in a high-risk context presumably exacts some perceived or actual professional costs. Given such a social setting, structural constraints of life-course position would be hypothesized to be more important than motivational factors in determining theory-choice behaviour. This implies, for example, that the earliest adopters of controversial theories should be disproportionately composed of middle-career and senior scientists and a corresponding deficit of young scientists.
In a low-risk context, a new theory is generally regarded as a legitimate claimant to knowledge, or one which has already attracted a sizeable following; consequently its adoption exposes one to only minimal professional costs. The social patterning of theory-choice behaviour in this context is hypothesized to be dominated by motivational factors, tending to reinforce more rapid adoption by younger scientists. [...]
During the early stages in the adoption of a new theory, age differences between supporters of the new theory and defenders of the status quo are expected to be either relatively small or (particularly if the new idea is perceived as being unusually controversial) tending toward older age for the first supporters. With the passage of time and greater acceptance of the new theory, we expect the influx of new converts to be increasingly drawn from the ranks of younger scientists. Such a correspondence between changes in the context of appraisal and age-based differences in theory choice is evident in McCann's data on French scientists' reception of the oxygen paradigm during the different subperiods of his study. It will be recalled that the earliest followers of oxygen theory were middle-aged scientists, and the greater propensity of younger scientists only became manifest several years later, at the point when community-wide conversion was well under way.
In the remainder of this paper, I present a rigorous test of the expanded age hypothesis proposed above. It is based upon new findings from a study of the reception of plate tectonics in the earth sciences during the 1960s. Compared with earlier studies, it permits a more precise delineation of the historical stages in prevailing scientific opinion, and introduces into the analysis controls on possible confounding factors correlated with age, such as foci of research interest and professional eminence. [...]
Development of the theory of plate tectonics ranks among the stellar scientific achievements of this century. General acceptance of this conceptual framework necessitated the abandonment of a communal belief in the horizontal immobility of the earth's crust which had guided geological research since the middle of the nineteenth century. Plate tectonics theory substituted the diametrically opposed premise that the earth's crus is divided into large crustal plates which move slowly over the upper mantle.
The swift adoption of plate tectonics during the late 1960s stands in stark contrast to the extremely bitter controversy encountered by earlier proponents of a 'mobile' earth. Alfred Wegener's continental drift theory, the forerunner of present-day mobilist theory, was subjected to extremely hostile attacks during the 1920s and fell into nearly universal disrepute. British geophysicists working on reconstruction of the ancient configurations of the earth's magnetic field rekindled interest in continental drift during the middle 1950s. [...] Despite the advocacy by Hess and a few other distinguished earth scientists, large-scale horizontal displacement of the crust remained an anathema for most earth scientists well into the 1960s. Then, in 1966-67, a confluence of empirical discoveries in marine geology, geomagnetic studies and seismology provided for many geologists incontrovertible proof favouring the seafloor spreading model, and the mobilist perspective more generally. [...] By the early 1970s, the great majority of earth scientists had adopted plate tectonics, and the theory was well on its way to becoming the dominant theoretical orientation in many fields of the earth sciences. [...]
To obtain data on the dynamics of individual theory choice that under earth scientists' shifts from the stabilist to the mobilist programme research, I examined the publications of ninety-six North American eart scientists actively engaged in pertinent research during the 1960s a early 1970s. I also gathered biographical information for each scientis including their training, research interests and career histories. [...]
The dependent variable for this study is the year in which a scientist decided to adopt the mobilist programme of research rather than to continue working within a stabilist programme. [...] Before 1966, when prevailing scientific opinion still ran strongly against the mobilist perspective, the small number of scientists adopting the programme were considerably older (in terms of career age) than other scientists active during this early period. Thus, scientists adopting the programme through 1963 were on average nineteen years 'older' than non-adopters. [...] Adopters in 1964 were twenty-three years older than non-adopters. [...] Only with the shift in scientific opinion favourable to mobilist concepts beginning in 1966, do we start to see a progressive narrowing, and then reversal, in the age differentials between adopters and non-adopters.
Interesting—this definitely suggests that Planck's statement probably shouldn't be taken literally/at face value if it is indeed true that some paradigm shifts have historically happened faster than generational turnover. It may still be possible that this may be measuring something slightly different than the initial 'resistance phase' that Planck was probably pointing at.
Two hesitations with the paper's analysis:
(1) by only looking at successful paradigm shifts, there might be a bit of a survival bias at play here (we're not hearing about the cases where a paradigm shift was successfully resisted and never came to fruition).
(2) even if senior scientists in a field may individually accept new theories, institutional barriers can still prevent that theory from getting adequate funding, attention, exploration. I do think Anthony's comment below nicely captures how the institutional/sociological dynamics in science seemingly differ substantially from other domains (in the direction of disincentivizing 'revolutionary' exploration).
It's neat to remember stories like this, but I want to note that this shouldn't necessarily update scientists to criticize novel work less. If an immune system doesn't sometimes overreact, it's not doing its job right, and for every story like this there are multiple other stories of genuinely false exciting-sounding ideas that got shut down by experts (for instance I learned about Schekhtman from the Constant podcast, where his story was juxtaposed with that of genuine quacks). Looking back at my experience of excited claims that were generally dismissed by more skeptical experts in fields I was following, the majority of them (for instance the superluminal neutrino, the room-temperature superconductor, various hype about potentially proving the Riemann hypothesis by well-established mathematicians) have been false.
I think there is a separate phenomenon (which was the explanation for the study about funerals), that older high-status scientists in funding-hungry fields will often continue to get funding and set priorities after they have stopped working on genuinely exciting stuff -- whether because of age, because of age-related conservatism bias, or simply because their area of expertise has become too well-developed to generate new ideas. In my experience in math and physics, from inside the field, this phenomenon generally does not look like a consensus that only the established people know what's going on (as in most of the stories here), but either conversely a quiet consensus that so-and-so famous person is starting to go crazy, or alternatively the normal disagreement between more conservative and more innovation-minded people about the value of a new idea. For example the most exciting development in my professional life as a mathematician was Jacob Lurie's development of "higher category theory", a revolution that allowed algebraists to seamlessly use tools from topology. There were many haters of this theory (many very young), but there was enough of a diffuse understanding that this is exciting and potentially revolutionary that his ideas did percolate and end up converting many of the haters (similarly with Grothendieck and schemes). Note that here I think math avoids the worst aspects of these dynamics because it doesn't require funding and is less competitive.
The upshot here is that I think it's valuable to try to resolve the issue of good ideas being shot down by traditionalists, but the solution might not be to "adopt lower standards for criticizing new / surprising ideas" but rather something more like pulling the rope sideways and looking for better standards that do better at separating promising innovation from hype.
1I think the issue here is not so much the disagreement or criticism as it is the mockery and ostracism. Unlike in, say, venture capital, there's much less opportunity in science for someone to try something different and exciting, get enough funding to see if it really works out, and then, if it doesn't but you were doing a good enough job trying, still be part of the community and get funding to try something else. (Yes, I know it doesn't always work that way in the startup community either, but I think the odds are much better than in science)
Thanks for this! Completely agree that there are Type I and II errors here and that we should be genuinely wary of both. Also agree with your conclusion that 'pulling the rope sideways' is strongly preferred to simply lowering our standards. The unconventional researcher-identification approach undertaken by the HHMI might be a good proof of concept for this kind of thing.
I am going to nit-pick on Wegener. His theory of continental drift is not plate tectonics, and he was wrong for pretty much all the reasons that other geologists and physicists of the time said he was wrong. Plate tectonics was able to explain Wegener crucial observation of the continents "fitting together" but with a different and plausible mechanism. His observation was an important and theory-driving anomaly. I remember a text book from 1960s examining both the strong evidence for continental matchup and the highly problematic issues with his idea of continent drift. An expanding earth was also postulated which fitted a lot of observation but would imply physical laws changed over long time periods. In short, it is a lot more nuanced. Similarly, Boltzmann's ideas on atomic theory were widely accepted in chemistry though physics took longer. Again, physics had an alternative hypothesis and it needed an experiment to separate them that didnt happen till after Boltzmann's death. I think there are similar nuances with Marshall and H Pyroli. The "heroic" lone scientist against the establishment may be an appealing narrative but in terms of how science actually makes progress, I think the nuances in these cases are important and more telling about the process.
by virtue of their technical chops, also care about their career capital.
I didn't understand this-- "their technical chops impose opportunity cost as they're able to build very safe successful careers if they toe the line" would make sense, or they care about career capital independent of their technical chops would make sense. But here, the relation between technical chops and caring about career capital doesn't come through clear.
Major scientific institutions talk a big game about innovation, but the reality is that many of the mechanisms designed to ensure quality—peer review, funding decisions, the academic hierarchy—explicitly incentivize incremental rather than revolutionary progress; are far more oriented towards 'day science' than 'night science.'[1]
Thomas Kuhn's now-famous notion of paradigm shifts was pointing at precisely this phenomenon. When scientists work within what Kuhn called "normal science," they're essentially solving low- to medium-stakes puzzles within their field's accepted framework. While it's fairly easy to evaluate the relative quality of work that occurs within any given paradigm, Kuhn argued it's nearly impossible for scientists to reason about the relative power of different paradigms for a given field—especially when they have already drank the paradigmatic kool-aid.
Max Planck captured this idea succinctly in his biting statement that "science advances one funeral at a time."[2]
There is no shortage of examples of this occurring throughout the history of science:
The takeaway here is quite relevant (and not all that unfamiliar) for alignment research. The still-young field attracts brilliant people who want to help solve the problem—and, by virtue of their technical chops, also care about their career capital. In attempting to check both of these boxes, many naturally gravitate toward "safer," already-somewhat-established research areas. However, when we polled these very researchers, most acknowledged they don't think these sorts of approaches will actually solve the core underlying problems in time. This seems quite familiar to the old story of incentives driving forward incremental work when what is desperately needed are breakthroughs.
The alignment innovations that will be most-critical-in-hindsight will have come from people who were willing to step outside the bounds, question the premises everyone took for granted, and pursue ideas that initially sounded ridiculous.
Got a crazy hunch that doesn't fit nicely into the current alignment landscape? Come talk to us at EAG Boston—or apply to work on your idea with us here.
This is not to say that incremental progress is unimportant or that revolutionary progress is all that matters—only that mainstream science is mostly in the business of operating under established paradigms rather than creating new ones.
While this is the better-remembered variant, Planck's actual statement was "a new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents die, and a new generation grows up that is familiar with it." But this would have made for too long a title.