I expect "slow takeoff," which we could operationalize as the economy doubling over some 4 year interval before it doubles over any 1 year interval. Lots of people in the AI safety community have strongly opposing views, and it seems like a really important and intriguing disagreement. I feel like I don't really understand the fast takeoff view.
(Below is a short post copied from Facebook. The link contains a more substantive discussion. See also: AI impacts on the same topic.)
I believe that the disagreement is mostly about what happens before we build powerful AGI. I think that weaker AI systems will already have radically transformed the world, while I believe fast takeoff proponents think there are factors that makes weak AI systems radically less useful. This is...
"Often I compare my own Fermi estimates with those of other people, and that’s sort of cool, but what’s way more interesting is when they share what variables and models they used to get to the estimate."
– Oliver Habryka, at a model building workshop at FHI in 2016
One question that people in the AI x-risk community often ask is
"By what year do you assign a 50% probability of human-level AGI?"
We go back and forth with statements like "Well, I think you're not updating enough on AlphaGo Zero." "But did you know that person X has 50% in 30 years? You should weigh that heavily in your calculations."
However, 'timelines' is not the interesting question. The interesting parts are in the causal models behind the estimates. Some possibilities:
This definitely occurs often, yet I think the assumption that status-driven shifts in beliefs are always sincere is false. Often we conform as we get penalised for deviation. Gibbard-Satterthwaite theorem implies that if there exists nondictatorship, independence of irrelevant alternatives, and Pareto unamity (the preferences of the majority aggregate to produce the social outcome preferred by the majority) then we will not always be sincere. You see this in poorly-designed auctions where people do not submit bits reflecting what they think the item is worth. I think when people skip step 2, often they haven't updated at all, but are conforming to not lose status.
I guess that was wrong