= 27d567bc2d48d9f40e9ccc20ea370b15)
Bets on an Extreme Future
Betting on the future is a good way to reveal true beliefs.
As one example of such a bet on a key debate about a post-human future, I'd like to announce here that Robin Hanson and I have made the following agreement. (See also Robin's post at Overcoming Bias):
It's a bet on the old question: ems vs. de novo AGI. Kurzweil and Kapor bet on another well-known debate: Will machines pass the Turing Test. It would be interesting to list some other key debates that we could bet on.
But it's hard to make a bet when settling the bet may occur in extreme conditions:
- after human extinction,
- in an extreme utopia,
- in an extreme dystopia or,
- after the bettors' minds have been manipulated in ways that redefine their personhood: copied thousands of times, merged with other minds, etc.
MIRI has a "techno-volatile" world-view: We're not just optimistic or pessimistic about the impact of technology on our future. Instead, we predict that technology will have an extreme impact, good or bad, on the future of humanity. In these extreme futures, the fundamental components of a bet--the bettors and the payment currency--may be missing or altered beyond recognition.
So, how can we calibrate our probability estimates about extreme events? One way is by betting on how people will bet in the future when they are closer to the events, on the assumption that they'll know better than we do. Though this is an indirect and imperfect method, it might be the best we have for calibrating our beliefs about extreme futures.
For example, Robin Hanson has suggested a market on tickets to a survival shelter as a way of betting on an apocalypse. However, this only relevant for futures where shelters can help; and where there is time to get to one while the ticket holder is alive, and while the social norm of honoring tickets still applies.
We could also define bets on the progress of MIRI and similar organizations. Looking back on the years since 2005, when I started tracking this, I would have liked to bet on, or at least discuss, certain milestones before they happened. They served as (albeit weak) arguments from authority or from social proof for the validity of MIRI's ideas. Some examples of milestones that have already been reached:
- SIAI's budget passing $500K per annum
- SIAI getting 4 full-time-equivalent employees
- SIAI publishing its fourth peer-reviewed paper
- The establishment of a university research center in relevant fields
- The first lecture on the core FAI thesis in an accredited university course
- The first article on the core FAI thesis in a popular science magazine
- The first mention of the core FAI thesis (or of SIAI as an organization) in various types of mainstream media, with a focus on the most prestigious (NPR for radio, New York Times for newspapers).
- The first (indirect/direct) government funding for SIAI
Looking to the future, we can bet on some other FAI milestones. For example, we could bet on these coming true by a certain year.
- FAI research in general (or: organization X) will have Y dollars in funding per annum (or: Z full-time researchers).
- Eliezer Yudkowsky will still be working on FAI.
- The intelligence explosion will be discussed on the floor of Congress (or: in some parliament; or: by a head of state somewhere in the world).
- The first academic monograph on the core FAI thesis will be published (apparently that will be Nick Bostrom's).
- The first master's thesis/PhD dissertation on the core FAI thesis will be completed.
- "Bill Gates will read at least one of 'Our Final Invention' or 'Superintelligence' in the next 2 years" (This already appears on PredictionBazaar.)
(Some of these will need more refinement before we can bet on them.)
Another approach is to bet on technology trends: brain scanning resolution; prices for computing power; etc. But these bets are about a Kurzweillian Law of Accelerating Returns, which may be quite distinct from the Intelligence Explosion and other extreme futures we are interested in.
Many bets only make sense if you believe that a soft takeoff is likely. If you believe that, you could bet on AI events while still allowing the bettors a few years to enjoy their winnings.
You can make a bet on hard vs. soft takeoff simply by setting your discount rate. If you're 20 years old and think that the economy as we know it will end instantly in, for example, 2040, then you won't save for your retirement. (See my article at H+Magazine.) But such decisions don't pin down your beliefs very precisely: Most people who don't save for their retirement are simply being improvident. Not saving makes sense if the human race is about to go extinct, but also if we are going to enter an extreme utopia or dystopia where your savings have no meaning. Likewise, most people save for retirement simply out of old-fashioned prudence, but you might build up your wealth in order to enjoy it pre-Singularity, or in order to take it with you to a post-Singularity world in which "old money" is still valuable.
I'd like to get your opinion: What are the best bets we can use for calibrating our beliefs about the extreme events we are interested in? Can you suggest some more of these indirect markers, or a different way of betting?
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
Comments (38)
Don't forget that individuals face a different set of extreme events than societies do. The biggest difference is death.
A personal bet on, say, an extinction event happening before 2040, is really a bet on (human extinction before 2040) OR (me dying before 2040).
"Control"? What if the humans "control" the machines?
This bet surely needs to compare machine brains with human brains. It isn't 100% clear that it is doing that.
This text is not the condition for the bet, but rather designates the point in time when we see how the conditions of the bet came out.
Apologies - in my distaste for "control" I misspoke.
I always worry about the legality of these types of bets. Does anyone else worry about that? Its not a big deal right now since they are rare and typically small, but the people who advocate for them seem to want them to be much more common.
It seems like there would also inevitably be ugly second-order effects if we had large betting markets on events with huge consequences for large numbers of people.
I mean, we know that gamblers will bribe or even injure sports competitors to win their bets. This is not a hypothetical. This happens, and for fairly small amounts of money. We know that business investors will try to move the markets, that they will pay for inside information, and that they will influence political campaigns and even destabilize governments.
Do we really want to give lots of people financial incentives for, say, a new flu outbreak?
We have them now. They are called financial markets.
Right, and for all the massive good they do in terms of capital allocation, they often do have ugly second order effects, even though they're only indirectly betting on whether the BCG vaccine can rebuild beta cells in Type 1 Diabetics, or Guinea's new government can be bribed into selling mineral rights cheaply at tremendous cost to their impoverished people.
Can you provide examples of social or economic systems which have "huge consequences for large numbers of people" and do NOT have ugly second order effects?
Nope. Everything does. So the questions are always how many and how ugly?
If financial markets are the most comparable system then you've got a promising data point. Nearly everybody will concede they've historically done a reasonable job of asset allocation even though they're haunted by bad actors and probably less functional now than ever. But if financial markets are already betting on the same sorts of things a betting market would, the betting market is redundant. It's not clear to me why you need a system of direct bets between individuals when you've already got hedge funds and GS spending collective billions on very smart people and special access to figure out which drug/weapon/nanotechnology/country is about to blow up, and then placing bets worth trillions. Obviously, the hedgies may not get it right, but they're less likely to be out of their depth than random gamblers with strong opinions.
If sports betting is the closest comparable system, then you've got a less promising data point. Sports betting always tends to corrupt the things it bets on because unlike asset allocation it's zero-sum or even negative-sum, and it's very hard to maintain a winning percentage without cheating, and cheating turns out to be remarkably easy. And even in the cleanest sorts of systems (the US is very clean from a global perspective) cabals with special access and tools conceal the actual flow of "smart money," which makes it very difficult to see what the market actually predicts.
Yep. Now let's reformulate that a bit. In the context of financial markets "betting" is called pricing. The price of an asset is the committed-capital-weighted average of the opinions of market participants.
So the question becomes, are the things we are interested in being priced in the financial markets.
Great, we're on the same track.
I submit that yes, many of the events LW/transhumanists/techno-futurists are interested in are being priced in financial markets right now. The financial markets are making guesses about robotics, nano- and materials tech, the prospect of nuclear exchanges, food insecurity, plagues, gene therapies, and other fancy ideas.
I'd also submit that many of the things not being priced in today's markets are extremely difficult to gain any info about, much less predict. They look much less like shares of companies or even CDOs, and much more like betting on soccer games.
I'd further submit--and I'd be willing to bet on this--that within a short period of creating a large incentive for catastrophe, wicked people will do something horrible to win a bet.
This is trivial to do now: buy oil futures and short oil companies, and blow up a pipeline. Oil-related contracts are notoriously volatile (amplifying gains) and such an attack would make a real difference. Yet despite John Robb's predictions that 'terrorist entrepreneurs' would fund their organizations this way, we have yet to see this.
But we do see behavior like this--not via spectacular attacks in the developed world, which would be extremely high risk, but via much more sophisticated, lower risk methods. Moving paper is much easier (and higher-reward) than violent crime. Spectacular attacks on oil facilities and assassinations of regulatory officials do occur, but they're usually in the developing world, much farther from cameras, and they're usually conducted by people who don't have slicker methods at hand.
And those are in capital markets, where people can make a profit without ever beating the average. Betting markets don't work that way. We always see behavior like this in betting markets. Competitors threatened or hurt or drugged, refs paid off or intimidated, bad information leaked to the public. Why would a betting market concerned with the political future look different than a betting market concerned with jai-alai?
I am not sure. How are financial markets pricing, say, the prospect of nuclear exchanges? Or plagues?
Well, that's just usually called crime -- unless a government does it in which case it's called policy. Wars have been started for financial gain, millions of people have been killed for political power, etc. etc. Are you suggesting there will be something new in this respect?
I'm suggesting that there's very little upside to the project compared to the downside.
Best case scenario, you get some very interested people with minimal information and strongly held beliefs betting on very long timelines with slow and irregular feedback to adjust their bets. It looks much more like the line on the World Cup than it does the price of Exxon, except that there's less historical data and less ability to adjust positions than you have in sports gambling. How much insight does this kind of betting market give you?
Worst case scenario, you get massive incentives for people to wish for disaster. Maybe you disagree that this leads to somebody dropping prions into cattle feed. Fair enough. But people do respond to incentives.
Edit to answer your first question regarding nuclear exchanges: Israel's stock market, and India's, move in response to investors' perceptions of Iran and Pakistan. Ditto everybody within the vicinity of the DPRK. Pharmaceutical companies' prices reflect investors' predictions about the likelihood of massive demand.
so wait, this bet is conditional on the singularity having pretty much occured, but you're betting simply cash? That sounds super boring. I would bet a genetically engineered alicorn steed or something.
Not cash per se, but rather "whatever $3000 invested today in S&P500-like funds today is worth then."
Can I say something radical?
A bet is only meaningful when the betting sides are expected to survive the event they are betting on.
In simpler words - if you are betting against your own death, you can't lose, because there won't be a you to lose.
Pointless definitional argument ahoy! Is losing a timeless relationship between you and the outcome, or is it the event of you coming to know that the outcome is against you?
Timeless relationships are for people who choose to live in the Block universe.
sticks tongue
Some more bets: