Of course.
But this harder to do than to say. Part of my brain would of course be relieved to discover that, say, I have less reason to expect human extinction this century than I had previously thought. On the other hand, I know that as crazy as it sounds, part of my brain is probably committed to my past stated beliefs and to my current mission, and would prefer to operate under confirmation bias.
Which means I need to do all kinds of things to be actually curious for the truth, like leaving a line of retreat for myself by visualizing what it would be like to make such a discovery and how the world would be okay with that result.
Which, since you gave me the opportunity, I just spent 20 seconds doing. :)
A few months ago, Anna Salamon and I began to write an academic overview of intelligence explosion scenarios — something we could hand to people to explain all our major points in one brief article.
We encountered two major problems.
First: The Summit happened, taking all of our time. Then I was made Executive Director, taking all of my time in a more persistent way.
Second: Being thorough and rigorous in an overview of intelligence explosion requires deep knowledge of a huge spectrum of science and philosophy: history of AI progress, history of planning for the future mattering, AI architectures, hardware progress, algorithms progress, massive datasets, neuroscience, factors in the speed of scientific progress, embryo selection, whole brain emulation, properties of digital minds, AI convergent instrumental values, self-improvement dynamics, takeoff scenarios, heuristics and biases, unipolar and multipolar intelligence explosion scenarios, human values and value extrapolation, decision theory, arms races, human dynamics of technological development, technological forecasting, the economics of machine intelligence, anthropics, evolution, AI-boxing, and much more. Because we were trying to write a short article, we kept having to consume and compress an entire field of knowledge into a single paragraph (or even a single sentence!) with the perfect 2-8 citations, which occasionally meant several days of work for a single paragraph. (This is an extreme example, but it's the kind of problem we often encountered, in different degrees.)
So, we've decided to take a different approach and involve the broader community.
We'll be posting short snippets, short pieces of the puzzle, for feedback from the community. Sometimes we'll pose questions, or ask for references about a given topic, or ask for suggested additions to the dialectic we present.
In the end, we hope to collect and remix the best and most essential snippets, incorporate the feedback and additions provided by the community, and write up the final article.
Think of it as a Polymath Project for intelligence explosion analysis. It's collaborative science and philosophy. Members of Less Wrong tend to be smart, and each one has deep knowledge of one or a few fields that we may not have. We hope you'll join us, and contribute your expertise to this project.
I'll keep a table of contents of all the snippets here, as they are published.
Draft #1:
Also see: