Maybe we're not doomed

Manfred

15 Maybe we're not doomed

by Manfred

2nd Aug 2014

2 min read

15

This is prompted by Scott's excellent article, Meditations on Moloch.

I might caricature (grossly unfairly) his post like this:

Map some central problems for humanity onto the tragedy of the commons.
Game theory says we're doomed.

Of course my life is pretty nice right now. But, goes the story, this is just a non-equilibrium starting period. We're inexorably progressing towards a miserable Nash equilibrium, and once we get there we'll be doomed forever. (This forever loses a bit of foreverness if one expects everything to get interrupted by self-improving AI, but let's elide that.)

There are a few ways we might not be doomed. The first and less likely is that people will just decide not to go to their doom, even though it's the Nash equilibrium. To give a totally crazy example, suppose there were two countries playing a game where the first one to launch missiles had a huge advantage. And neither country trusts the other, and there are multiple false alarms - thus pushing the situation to the stable Nash equilibrium of both countries trying to launch first. Except imagine that somehow, through some heroic spasm of insanity, these two countries just decided not to nuke each other. That's the sort of thing it would take.

Of course, people are rarely able to be that insane, so success that way should not be counted on. But on the other hand, if we're doomed forever such events will eventually occur - like a bubble of spontaneous low entropy spawning intelligent life in a steady-state universe.

The second and most already-implemented way is to jump outside the system and change the game to a non-doomed one. If people can't share the commons without defecting, why not portion it up into private property? Or institute government regulations? Or iterate the game to favor tit-for-tat strategies? Each of these changes has costs, but if the wage of the current game is 'doom,' each player has an incentive to change the game.

Scott devotes a sub-argument to why we're still doomed to things be miserable if we solve coordination problems with government:

Incentives for government employees sometimes don't match the needs of the people.
This has costs, and those costs help explain why some things that suck, suck.

I agree with this, but not all governments are equally costly as coordination technologies. Heck, not all governments even are a technology for improving peoples' lives - look at North Korea. My point is that there's no particular reason that costs can't be small, with sufficiently advanced cultural technology.

More interesting to me than government is the idea of iterating a game to to encourage cooperation. In the normal prisoner's dilemma game, the only Nash equilibrium is defect-defect and so the prisoners are doomed. But if you have to play the prisoner's dilemma game repeatedly, with a variety of other players, the best strategy turns out to be a largely cooperative one. This evasion of doom gives every player an incentive to try and replace one-shot dilemmas with iterated ones. Could Scott's post look like this?

Map some central problems for humanity onto the iterated prisoner's dilemma.
Evolutionary game theory says we're not doomed.

In short, I think this idea of "if you know the Nash equilibrium sucks, everyone will help you change the game" is an important one. Though given human irrationality, game-theoretic predictions (whether of eventual doom or non-doom) should be taken less than literally.

Personal Blog

15

New Comment

Rendering 0/47 comments, sorted by

top scoring

(show more) Click to highlight new comments since: Today at 9:49 PM

Moderation Log

15 Maybe we're not doomed

by Manfred

2nd Aug 2014

2 min read

15

This is prompted by Scott's excellent article, Meditations on Moloch.

I might caricature (grossly unfairly) his post like this:

Map some central problems for humanity onto the tragedy of the commons.
Game theory says we're doomed.

Scott devotes a sub-argument to why we're still doomed to things be miserable if we solve coordination problems with government:

Incentives for government employees sometimes don't match the needs of the people.
This has costs, and those costs help explain why some things that suck, suck.

Map some central problems for humanity onto the iterated prisoner's dilemma.
Evolutionary game theory says we're not doomed.

Personal Blog

15

New Comment

Rendering 0/47 comments, sorted by

top scoring

(show more) Click to highlight new comments since: Today at 9:49 PM

Moderation Log

More from Manfred

Curated and popular this week

47Comments

Comment Permalink

pianoforte61112y60

Scott's post proves too much. In particular it proves that early multicellular organisms would inevitably revert back to micro-organisms.

Edit: This is a bit too opaque. Scott argues that agents that sacrifice the common good for individual gains in reproduction and survival sill survival will outcompete agents that do not make these sacrifices. However, cells within early multicellular organisms could have done this, yet multicellular organisms nevertheless exist.

Viliam_Bur12y100

I understood it to mean that multicellular organisms don't maximize the values of their cells, if we would anthropomorphize them.

In other words, in this scenario you are the (potential) cancer. And the system evolves in a way that you either lose your battles (most of the time), or your victory against the system would result in your death anyway. (Because your survival depends on the system in too many ways.)

9roystgnr12y

Look at example 9: cancer. That basically is a multicellular organism reverting back to micro-organisms. And in some sense it's as depressing as Scott says: billions of years, and evolution still hasn't figured out an immune system that can reliably put an end to that garbage? In another sense it's outright encouraging: billions of years of animals afflicted by cancer, and here we still are, and that wasn't just good luck. The winning strategy seems to be "group selection", where we think of organisms as a group of cooperating cells. Cancer may eventually take my group of cells down, but if by the time that happens there's two or three independent groups running around preserving many of the same values (in this case genes, memes, etc) as my group had, those values live on. In a third sense it's much more depressing. Scott's only identified solution category is to prevent runaway competition from destroying our values by governing all that competition with a singleton: a single agent or coordinated group so powerful as to be able to squelch any outsider or subgroup's attempt to subvert human values, even if that villain is specifically making itself as effective as possible by sacrificing those values on the altar of efficiency. But in the cancer analogy, this is the sort of idea that would have ended up with the death of all multicellular life: even if a single megaorganism might be vastly better than run-of-the-mill organisms at defeating cancer (better immune systems via economies of scale, less energy wasted in zero-sum games against competing organisms, whatever), it only needs to lose once and it's lost for all time. Back outside the analogy, either Scott makes his singleton perfectly stable, or it gets replaced by whatever manages to subvert it, and now whatever managed to subvert it is a singleton. FAI slips up and we get a universe tiled with smiley faces. Archipelago slips up and we get a boot stamping on a human face forever.

2[anonymous]12y

Additionally, if the history of life on Earth should show you anything its that nothing ever 'wins'.

See in context