For big jumpy events, look at the reactions to nuclear chain reactions, Sputnik, ENIGMA, penicillin, the Wright brothers, polio vaccine...
Then consider the process of gradual change with respect to the Internet, solar power, crop yields...
I do tend to think that Aubrey de Grey's argument holds some water. That is, it's not so much general society that will be influenced as wealthy elites. Elites seem more likely to update when they read about a 2x mouse. I suppose the Less Wrong response to this argument would be: how many of them are signed up for cryonics? But cryonics is a lot harder to believe than life extension. You need to buy pattern identity theory and nanotechnology and Hanson's value of life calculations. In the case of LE, all you have to believe is that the techniques that worked on the mouse will, likely, be useful in treating human senescence. And anyway, Aubrey hopes to first convince the gerontology community and then the public at large. This approach has worked for climate science and a similar approach may work for AI risk.
I suppose the Less Wrong response to this argument would be: how many of them are signed up for cryonics?
LessWrongers, and high-karma LessWrongers, on average seem to think cryonics won't work, with mean odds of 5:1 or more against cryonics (although the fact that they expect it to fail doesn't stop an inordinate proportion from trying it for the expected value).
On the other hand, if mice or human organs were cryopreserved and revived without brain damage or loss of viability, people would probably become a lot more (explicitly and emotionally) confident that there is no severe irreversible information loss. Much less impressive demonstrations have been enough to create huge demand to enlist in clinical trials before.
That number is the total probability of being revived taking into account x-risk among other things. It would be interesting to know how many people think it's likely to be technically feasable to revive future cryo patients.
I don't find either example convincing about the general point. Since I'm stupid I'll fail to spot that the mouse example uses fictional evidence and is best ignored
We are all pretty sick of seeing a headline "Cure for Alzheimer's disease!!!" and clicking through to the article only to find that it is cured in mice, knock-out mice, with a missing gene, and therefore suffering from a disease a little like human Alzheimer. The treatment turns out to be injecting them with the protein that the missing gene codes for. Relevance to human health: zero.
Mice are very short lived. We expect big boosts in life span by invoking mechanisms already present in humans and already working to provide humans with much longer life spans than mice. We don't expect big boosts in the life span of mice to herald very much for human health. Cats would be different. If pet cats started living 34 years instead of 17, their owners would certainly be saying "I want what Felix is getting."
The sophistication of AI is a tricky thing to measure. I think that we are safe from unfriendly AI for a few years yet, not so much because humans suck at programming computers, but because they suck in a ...
As far as I know, people have predicted every single big economic impact from technology well in advance, in the strong sense of making appropriate plans, making indicative utterances, etc. (I was claiming a few year's warning in the piece you are responding to, which is pretty minimal). Do you think there are counterexamples? You are claiming that a completely unprecedented will happen with very high probability. If you don't think that requires strong arguments to justify than I am confused, and if you think you've provided strong arguments I'm confused too.
I agree that AI has the potential to develop extremely quickly, in a way that only a handful of other technologies did. As far as I can tell the best reason to suspect that AI might be a surprise is that it is possible that only theoretical insights are needed, and we do have empirical evidence that sometimes people will be blindsided by a new mathematical proof. But again, as far as I know that has never resulted in a surprising economic impact, not even a modest one (and even in the domain of proofs, most of them don't blindside people, and there are strong arguments that AI is a harder problem than the problems that one per...
You mention Deep Blue beating Kasparov. This sounds look a good test case. I know that there were times when it was very controversial whether computers would ever be able to beat humans in chess - Wikipedia gives the example of a 1960s MIT professor who claimed that "no computer program could defeat even a 10-year-old child at chess". And it seems to me that by the time Deep Blue beat Kasparov, most people in the know agreed it would happen someday even if they didn't think Deep Blue itself would be the winner. A quick Google search doesn't pull up enough data to allow me to craft a full narrative of "people gradually became more and more willing to believe computers could beat grand masters with each incremental advance in chess technology", but it seems like the sort of thing that probably happened.
I think the economics example is a poor analogy, because it's a question about laws and not a question of gradual creeping recognition of a new technology. It also ignores one of the most important factors at play here - the recategorization of genres from "science fiction nerdery" to "something that will happen eventually" to "something that might happen in my lifetime and I should prepare for it."
I know that there were times when it was very controversial whether computers would ever be able to beat humans in chess
Douglas Hofstadter being one on the wrong side: well, to be exact, he predicted (in his book GEB) that any computer that could play superhuman chess would necessarily have certain human qualities, e.g., if you ask it to play chess, it might reply, "I'm bored of chess; let's talk about poetry!" which IMHO is just as wrong as predicting that computers would never beat the best human players.
I thought you were exaggerating there, but I looked it up in my copy and he really did say that: pg684-686:
...To conclude this Chapter, I would like to present ten "Questions and Speculations" about AI. I would not make so bold as to call them "Answers" - these are my personal opinions. They may well change in some ways, as I learn more and as AI develops more...
Question: Will there be chess programs that can beat anyone?
Speculation: No. There may be programs which can beat anyone at chess, but they will not be exclusively chess players. They will be programs of general intelligence, and they will be just as temperamental as people. "Do you want to play chess?" "No, I'm bored with chess. Let's talk about poetry." That may be the kind of dialogue you could have with a program that could beat everyone. That is because real intelligence inevitably depends on a total overview capacity - that is, a programmed ability to "jump out of the system", so to speak - at least roughly to the extent that we have that ability. Once that is present, you can't contain the program; it's gone beyond that certain critical point, and you just have to face
I suspect the thermostat is closer to the human mind than his conception of the human mind is.
I notice I am confused (he said politely). Kasparov is not stupid and modern chess programs on a home computer e.g. Deep Rybka 3.0 are overwhelmingly more powerful than Deep Blue, there should be no reasonable way for anyone to delude themselves that computer chess programs are not crushingly superior to unassisted humans.
Example 1: "After a 2-year-old mouse is rejuvenated to allow 3 years of additional life, society will realize that human rejuvenation is possible, turn against deathism as the prospect of lifespan / healthspan extension starts to seem real, and demand a huge Manhattan Project to get it done."
A quick and dirty Google search reveals:
Cost of Manhattan Project in 2012 dollars: 30 billion
Pharma R&D budget in 2012: 70 billion
http://www.fiercebiotech.com/special-reports/biopharmas-top-rd-spenders-2012
http://nuclearsecrecy.com/blog/2013/05/17/the-price-of-the-manhattan-project/
My version of Example 2 sounds more like "at some point, Watson might badly misdiagnose a human patient, or a bunch of self-driving cars might cause a terrible accident, or more inscrutable algorithms will do more inscrutable things, and this sort of thing might cause public opinion to turn against AI entirely in the same way that it turned against nuclear power."
In general and across all instances I can think of so far, I do not agree with the part of your futurological forecast in which you reason, "After event W happens, everyone will see the truth of proposition X, leading them to endorse Y and agree with me about policy decision Z."
Sir Karl Popper came to the same conclusion in his 1963 book Conjectures and Refutations. So did Harold Walsby in his 1947 book The Domain of Ideologies. You're in good company.
At the Edge question 2009 6 people spoke of immortality (de Grey not included). and 17 people spoke of superintelligence/humans 2.0.
This seems like evidence for Aubrey's point of view.
Of all the things that 151 top scientists could think they'd live to see, that more than 10% converged on that without previous communication in that stuff is perplexing for anyone who was a transhumanist in 2005.
without previous communication
No, these people all have long term relationships with Brockman/Edge, which even holds parties bringing them together.
I agree that almost no actual individual will change his or her mind. But humanity as a whole can change its mind, as young impressionable scientists look around, take in the available evidence, and then commit themselves to a position and career trajectory based on that evidence.
I do think there is a lot of truth to that. Reminds me of the people who said in the 1990's "Well, as soon as the arctic ice cap start to melt, then the climate deniers will admit that climate change is real", but of course that haven't happened either.
I do wonder, though, if that's equally true for all of those fields. For example, in terms of anti-aging technology, it seems to me that the whole status quo is driven by a very deep and fundamental sense that aging is basically unchangeable, and that that's the only thing that makes it accepta...
"When the author of the original data admits that he fabricated it, people will stop believing that vaccines cause autism."
Funny that I just hit this with my comment from yesterday:
http://lesswrong.com/lw/hoz/do_earths_with_slower_economic_growth_have_a/95hm
Idea: robots can be visibly unfriendly without being anywhere near a FOOM. This will help promote awareness.
I think this is different from your examples.
A mouse living to the ripe old age of 5? Well, everyone has precedent for animals living that long, and we also have precedent for medicine doing all sorts of amazing things for the health of mice that seem never to translate into treatments for people.
Economic meltdowns? ...
I wonder how a prolonged event Y, something with enough marketability to capture the public's eye for some time, might change opinions on the truth of proposition X. Something along the lines of the calculus, Gutenberg's printing press, the advent of manoeuvrable cannons, flintlocks, quantum mechanics, electric stoves (harnessed electricity), the concept of a national debt, etcetera.
I'd be interested in what if any effects the American National Security Agency scandal, or a worldwide marketing campaign by Finland advertising their healthcare system will or would have to this end.
Presumably the best you can do solution-wise is to try and move policy options through a series of "middle stages" towards either optimal results, or more likely the best result you can realistically get?
EDIT: Also- how DID the economists figure it out anyway? I would have thought that although circumstances can incease or reduce it inflationary effects would be inevitable if you increased the money supply that much.
The two examples here seem to not have alarming/obvious enough Ws. It seems like you are arguing against a straw-man who makes bad predictions, based on something like a typical mind fallacy.
Making a mouse live for 5 years isn't going to get anyone's attention. When they can make a housecat live to be 60, then we'll talk.
unrestricted Turing test passing should be sufficient unto FOOM
I don't think this is quite right. Most humans can pass a Turing test, even though they can't understand their own source code. FOOM requires that an AI has the ability to self-modify with enough stability to continue to (a) desire to continue to self-modified, and (b) be able to do so. Most uploaded humans would have a very difficult time with this - - just look at how people resist even modifying their beliefs, let alone their thinking machinery.
The problem is that an AI which passes the unrestricted Turing test must be strictly superior to a human; it would still have all the expected AI abilities like high-speed calculation and so on. A human who was augmented to the point of passing the Pocket Calculator Equivalence Test would be superhumanly fast and accurate at arithmetic on top of still having all the classical human abilities, they wouldn't be just as smart as a pocket calculator.
High speed calculation plus human-level intelligence is not sufficient for recursive self-improvement. An AI needs to be able to understand its own source code, and that is not a guarantee that passing the Turing test (plus high-speed calculation) includes.
Exactly. This is part of the reason I will win the bet, i.e. it is the reason the first super intelligent AI will be programmed without attention to Friendliness.
if you look at what happens when Congresscritters question Bernanke, you will find that they are all terribly, terribly concerned about inflation
I imagine that Congress is full of busy people who don't have time to follow blogs that cover every topic they debate. Did they get any testimony from expert economists when questioning Bernanke?
"After event W happens, everyone will see the truth of proposition X, leading them to endorse Y and agree with me about policy decision Z."
Isn't this just a standard application of Bayesianism? I.e. after event W happens, people will consider proposition X to be somewhat more likely, thereby making them more favorable to Y and Z. The stronger evidence event W is, the more people will update and the further they will update. But no one piece of evidence is likely to totally convince everyone immediately, nor should it.
For instance, if "a 2-y...
I'd missed the mouse "rejuvenation for 3 more years of life" result (did you mean cryo freeze -> revive, or something else?). Could you supply a cite?
Aubrey de Grey thinks that it's worthwhile to fund a big price for the first group who achieves that result. That's one of the main strategies he advocates to convince everyone to take aging seriously.
In general and across all instances I can think of so far, I do not agree with the part of your futurological forecast in which you reason, "After event W happens, everyone will see the truth of proposition X, leading them to endorse Y and agree with me about policy decision Z."
Example 1: "After a 2-year-old mouse is rejuvenated to allow 3 years of additional life, society will realize that human rejuvenation is possible, turn against deathism as the prospect of lifespan / healthspan extension starts to seem real, and demand a huge Manhattan Project to get it done." (EDIT: This has not happened, and the hypothetical is mouse healthspan extension, not anything cryonic. It's being cited because this is Aubrey de Grey's reasoning behind the Methuselah Mouse Prize.)
Alternative projection: Some media brouhaha. Lots of bioethicists acting concerned. Discussion dies off after a week. Nobody thinks about it afterward. The rest of society does not reason the same way Aubrey de Grey does.
Example 2: "As AI gets more sophisticated, everyone will realize that real AI is on the way and then they'll start taking Friendly AI development seriously."
Alternative projection: As AI gets more sophisticated, the rest of society can't see any difference between the latest breakthrough reported in a press release and that business earlier with Watson beating Ken Jennings or Deep Blue beating Kasparov; it seems like the same sort of press release to them. The same people who were talking about robot overlords earlier continue to talk about robot overlords. The same people who were talking about human irreproducibility continue to talk about human specialness. Concern is expressed over technological unemployment the same as today or Keynes in 1930, and this is used to fuel someone's previous ideological commitment to a basic income guarantee, inequality reduction, or whatever. The same tiny segment of unusually consequentialist people are concerned about Friendly AI as before. If anyone in the science community does start thinking that superintelligent AI is on the way, they exhibit the same distribution of performance as modern scientists who think it's on the way, e.g. Hugo de Garis, Ben Goertzel, etc.
Consider the situation in macroeconomics. When the Federal Reserve dropped interest rates to nearly zero and started printing money via quantitative easing, we had some people loudly predicting hyperinflation just because the monetary base had, you know, gone up by a factor of 10 or whatever it was. Which is kind of understandable. But still, a lot of mainstream economists (such as the Fed) thought we would not get hyperinflation, the implied spread on inflation-protected Treasuries and numerous other indicators showed that the free market thought we were due for below-trend inflation, and then in actual reality we got below-trend inflation. It's one thing to disagree with economists, another thing to disagree with implied market forecasts (why aren't you betting, if you really believe?) but you can still do it sometimes; but when conventional economics, market forecasts, and reality all agree on something, it's time to shut up and ask the economists how they knew. I had some credence in inflationary worries before that experience, but not afterward... So what about the rest of the world? In the heavily scientific community you live in, or if you read econblogs, you will find that a number of people actually have started to worry less about inflation and more about sub-trend nominal GDP growth. You will also find that right now these econblogs are having worry-fits about the Fed prematurely exiting QE and choking off the recovery because the elderly senior people with power have updated more slowly than the econblogs. And in larger society, if you look at what happens when Congresscritters question Bernanke, you will find that they are all terribly, terribly concerned about inflation. Still. The same as before. Some econblogs are very harsh on Bernanke because the Fed did not print enough money, but when I look at the kind of pressure Bernanke was getting from Congress, he starts to look to me like something of a hero just for following conventional macroeconomics as much as he did.
That issue is a hell of a lot more clear-cut than the medical science for human rejuvenation, which in turn is far more clear-cut ethically and policy-wise than issues in AI.
After event W happens, a few more relatively young scientists will see the truth of proposition X, and the larger society won't be able to tell a damn difference. This won't change the situation very much, there are probably already some scientists who endorse X, since X is probably pretty predictable even today if you're unbiased. The scientists who see the truth of X won't all rush to endorse Y, any more than current scientists who take X seriously all rush to endorse Y. As for people in power lining up behind your preferred policy option Z, forget it, they're old and set in their ways and Z is relatively novel without a large existing constituency favoring it. Expect W to be used as argument fodder to support conventional policy options that already have political force behind them, and for Z to not even be on the table.