"Continental drift" is usually the go-to example. For one, the mechanism originally proposed was complete nonsense...
There was a pretty solid basis for believing that 2-dimensional crystals were thermodynamically unstable and thus couldn't exist. Then in 2004 Geim and Novoselov did it (isolated graphene for the first time) and people had to re-scrutinize the theory, since it was obviously wrong somehow. It turns out that the previous theory was correct for 2D crystals of essentially infinite size, but it seems to not apply for non-infinite crystals. At least that is how it was explained to me once by a theorist on the subject.
The opening paragraph of this paper cites the relevant literature: http://cdn.intechopen.com/pdfs/40438/InTech-The_cherenkov_effect_in_graphene_like_structures.pdf
Single-layer Graphene is really really unstable and if you let it sit free, readily scrolls up and is very hard to get unstuck. In this sense, Landau's impossibility proof is entirely correct.
And that's why we don't use free-standing graphene without a frame, for just about anything. The closest we get is graphene oxide dissolved in a liquid, or extremely extremely tiny platelets that don't really deserve to be called crystals.
The pessimism about non-usefulness of graphene lay entirely in forgetting that you could put it on a backing or stretch it out (or thinking that it would lose its interesting properties if you did the former), and that was not justifiable at all.
Lord Kelvin was wrong but was he pessimistic? He wasn't saying we could never know the answer, or visit the sun, or anything like that. Yes, he guessed wrongly, and too low, but it doesn't seem to be the case that 'underestimating a quantity' is pessimism. If nothing else, the quantity might be 'number of babies killed'.
The claim that the Sun revolves around the Earth. If the Earth revolved around the Sun, there would have been a parallax in the observations of stars from different positions in the orbit. There was no observable parallax, so Earth probably didn't revolve around the Sun.
Off the top of my head, how about the Landau Pole? A famous and usually right genius calculated that the gauge theories of quantum fields are a dead end, and set the Soviet and to some degree Western physics a few years back, if I recall correctly. His calculation was not wrong, he simply missed the alternate possibilities.
EDIT: hmm, I'm having trouble locating any links discussing the negative effects of the Landau pole discovery on the QED research.
We also know many famous examples of scientists just completely making up their pessimism, for example about the impossibility of human heavier-than-air flight.
This isn't what you asked for, but I might as well enumerate a few of these examples, for everyone's benefit. For the field of AI research:
"You can build a machine to draw [logical] conclusions for you, but I think you can never build a machine that will draw [probabilistic] inferences."
George Pólya (1954), ch. 15 — a few decades before the probabilistic revolution in AI.
...[Machines
[Machines] cannot play chess any more than they can play football.
Technically, he was correct.
Taube did not mean "Machines cannot be made to choose good chess moves" (a claim that has, indeed, been amply falsified). Here's a bit more context, from the linked paper.
[...] there are analog relationships in real chess -- such as the emptiness of a line [...] which cannot be directly handled by any digital machine. These analog relationships can be approximated digitally [...] in order to determine whether a given line is empty [...] such a set of calculations is not identical to the visual recognition that the space between two pieces is empty. A large part of the enjoyment of chess [...] derives from its deployment or topological character, which a machine cannot handle except by elimination. If game is used in the usual sense -- that is, as it was used before the word was redefined by computer enthusiasts with nothing more serious to do -- it is possible to state categorically that machines cannot play games. They cannot play chess any more than they can play football.
Taube's point, if I'm not misunderstanding him grossly, is that part of what it means to play a game of chess is (not merely to choose moves repeatedly until the game is over, but) to have somethin...
You accuse lukeprog of being misleading in taking a quote from a mere "librarian", and as we all know, a librarian is a harmless drudge who just shelves books, hence
it doesn't confer the kind of expertise that would make it surprising or even very interesting for Taube to have been wrong here.
I accuse you of being highly misleading in at least two ways here:
Mortimer Taube turns out to be the kind of 'librarian' who exemplifies this; the little byline to his letter about "Documentation Incorporated" should have been an indicator that maybe he was more than just a random schoolhouse librarian stamping in kids' books, but because you did not see fit to add any background on what sort of 'librarian' Taube was, I will:
...He is on the list of the 100 most important leaders in Library and I
Here is another famous example:Chandrasekhar's limit. Eddington rejected the idea of black holes ("I think there should be a law of Nature to prevent a star from behaving in this absurd way!"). Says wikipedia:
Chandra's discovery might well have transformed and accelerated developments in both physics and astrophysics in the 1930s. Instead, Eddington's heavy-handed intervention lent weighty support to the conservative community astrophysicists, who steadfastly refused even to consider the idea that stars might collapse to nothing.
I guess this ...
The general success rate of breakthroughs is pretty damn low, and so I'd argue that most examples of "invalid" pessimism (excluding some stupid ones coming from scientists you never heard of before coming across a quote, and excluding things like PR campaigning by Edison), viewed in the context of almost all breakthroughs failing for some reason you can't anticipate, are not irrational but simply reflect absence of strong evidence in favour of success (and absence of strong evidence against unknown obstacles), at the time of assessment (and corre...
I'm not sure if this is justifiable or just an old-fashioned blunder...
On the subject of stars, all investigations which are not ultimately reducible to simple visual observations are…necessarily denied to us… We shall never be able by any means to study their chemical composition.
-- August Comte, 1835
I'm leaning towards "blunder" myself...
Yeah, blunder. Wikipedia says:
In the 1820s both John Herschel and William H. F. Talbot made systematic observations of salts using flame spectroscopy. In 1835, Charles Wheatstone reported that different metals could be easily distinguished by the different bright lines in the emission spectra of their sparks, thereby introducing an alternative mechanism to flame spectroscopy.
it has been plausibly argued to me that all the roads to nuclear weapons, including plutonium production from U-238, may have bottlenecked through the presence of significant amounts of Earthly U235
This has interesting repercussions for Fermi's paradox.
I posted the following in a quotes page a few months back. I don't know how justifiable these were, and these are only questionably pessimism, but there may be some interesting examples in this. In particular, my light knowledge of the subject suggests that there really were extremely compelling reasons to disregard Feynman's formulation of QED for many years after it was first introduced.
...It is interesting to note that Bohr was an outspoken critic of Einstein's light quantum (prior to 1924), that he mercilessly denounced Schrodinger's equation, discourag
Here's an example of the 'opposite' - a case of unjustifiable correct optimism:
Columbus knew the Earth was round but should also have known the radius of the Earth and size of Eurasia well enough to know that the voyage East to Asia was simply impossible with the ships and supplies he went with. It seems to have turned out OK for him, though.
This is probably not a very useful example and I wouldn't be surprised to see that there were plenty more of these examples.
Kuhn's Structure of Scientific Revolutions is all about how an old scientific approach is often more right than the new school -- fits the data better, at least in the areas widely acknowledged to be central. Only later does the new approach become refined enough to fit the data better.
Thomas Malthus' view that in the long run we will always be stuck in (what we now call) the Malthusian trap. He would have been right if not for the sustained growth given to us by the industrial revolution.
it has been plausibly argued to me that all the roads to nuclear weapons, including plutonium production from U-238, may have bottlenecked through the presence of significant amounts of Earthly U235 (apparently even the giant heap of unrefined uranium bricks in Chicago Pile 1 was, functionally, empty space with a scattering of U235 dust).
All is such a strong word unless supplemented with qualifiers. I question the plausibility the arguments at supporting that absolute. The route "wait for an extra century or two of particle physics research and spend a few trillion producing the initial seed stock" would still be available.
In context, Fermi was considering something rather more short-term: WW2.
That said, he may not have scoped his statement to such a small scale.
You accuse lukeprog of being misleading in taking a quote from a mere "librarian", and as we all know, a librarian is a harmless drudge who just shelves books
I really can't think of a polite way to say this, so:
Bullshit.
I wasn't accusing Luke of anything; I was disagreeing with him. Disagreement is not accusation. When I want to make an accusation, I will make an accusation, like this one: You have mischaracterized what I wrote, and made totally false insinuations about my opinions and attitudes, and I have to say I'm pretty shocked to see someone as generally excellent as you behaving in such a way.
I do not think, and I did not say, and I had not the slightest intention of implying, that "a librarian is a harmless drudge who just shelves books".
Allow me to remind you how Luke's comment begins. The boldface emphasis is mine.
We also know many famous examples of scientists just completely making up their pessimism, for example about the impossibility of human heavier-than-air flight.
This isn't what you asked for, but I might as well enumerate a few of these examples, for everyone's benefit. For the field of AI research:
Taube was, despite his many excellent qualities, not a scientist as that term is generally understood, and he was, despite his many excellent qualities, not working in "the field of AI research".
(Yes, I know the Wikipedia page says he was "a true innovator in the field of science". Reading what it says he did, though, I really can't see that what he did was science. For the avoidance of doubt, and in the probably overoptimistic hope that saying this will stop you pulling the same what-a-snob-this-person-is move as you already did above, I don't think that "not science" is in any way the same sort of thing as "not valuable" or "not important" or "not difficult". What the creators of (say) the Firefox web browser did was important and valuable and difficult, but happens not to be science. What Beethoven did was important and valuable and difficult, but happens not to be science. What Martin Luther King did was important and valuable and difficult, but happens not to be science.)
Pointing this out doesn't mean I think there's anything wrong with being a librarian. When I said "a librarian is a fine thing to be", I meant it. (And, for the avoidance of doubt, it is my opinion both when "librarian" means "someone who shelves books in a library" and when it means "a world-class expert on organizing information in catalogues".)
Now, having said all that, I should add that you are quite right about one thing: when I said that Taube was neither a computer expert nor a philosopher, I was oversimplifying. (Not least because I hadn't looked deeply into Taube's career.) He was an important innovator in the use of punched cards for document indexing, which is quite a bit like being a computer expert; and he was a PhD in philosophy, which is quite a bit like being a philosopher. None the less, I stand by what I said: neither being a world-class expert in document indexing, nor knowing a lot about punched-card reading machinery, nor being a PhD in philosophy, seems to me to be the kind of expertise that makes it particularly startling if one's wrong about whether machines can play chess.
(And, once again, for the avoidance of doubt, I am not in the least trying to belittle his expertise and creativity. I just don't see that they were the kind of expertise and creativity that make it startling for someone to be wrong about the possibilities of computer chess-playing.)
[EDITED to clarify a bit of wording and add some emphasis. ... And again, later, to add a missing negative; oops. Also, while I'm here, two other remarks. 1: I regret the confrontational tone this exchange has taken; but I don't see any way I could have responded sufficiently forcefully to the accusations levelled at me without perpetuating it. 2: I see a lot of downvotes are flying around in this subthread. For the record, I haven't cast any.]
In an erratum to my previous post on Pascalian wagers, it has been plausibly argued to me that all the roads to nuclear weapons, including plutonium production from U-238, may have bottlenecked through the presence of significant amounts of Earthly U235 (apparently even the giant heap of unrefined uranium bricks in Chicago Pile 1 was, functionally, empty space with a scattering of U235 dust). If this is the case then Fermi's estimate of a "ten percent" probability of nuclear weapons may have actually been justifiable because nuclear weapons were almost impossible (at least without particle accelerators) - though it's not totally clear to me why "10%" instead of "2%" or "50%" but then I'm not Fermi.
We're all familiar with examples of correct scientific skepticism, such as about Uri Geller and hydrino theory. We also know many famous examples of scientists just completely making up their pessimism, for example about the impossibility of human heavier-than-air flight. Before this occasion I could only think offhand of one other famous example of erroneous scientific pessimism that was not in defiance of the default extrapolation of existing models, namely Lord Kelvin's careful estimate from multiple sources that the Sun was around sixty million years of age. This was wrong, but because of new physics - though you could make a case that new physics might well be expected in this case - and there was some degree of contrary evidence from geology, as I understand it - and that's not exactly the same as technological skepticism - but still. Where there are sort of two, there may be more. Can anyone name a third example of erroneous scientific pessimism whose error was, to the same degree, not something a smarter scientist could've seen coming?
I ask this with some degree of trepidation, since by most standards of reasoning essentially anything is "justifiable" if you try hard enough to find excuses and then not question them further, so I'll phrase it more carefully this way: I am looking for a case of erroneous scientific pessimism, preferably about technological impossibility or extreme difficulty, where it seems clear that the inverse case for possibility would've been weaker if carried out strictly with contemporary knowledge, after exploring points and counterpoints. (So that relaxed standards for "justifiability" will just produce even more justifiable cases for the technological possibility.) We probably should also not accept as "erroneous" any prediction of technological impossibility where it required more than, say, seventy years to get the technology.