You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Romashka comments on Roadmap: Plan of Action to Prevent Human Extinction Risks - Less Wrong Discussion

13 Post author: turchin 01 June 2015 09:58AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (88)

You are viewing a single comment's thread. Show more comments above.

Comment author: Satoshi_Nakamoto 12 June 2015 02:39:09PM *  2 points [-]

I would use the word resilient rather than robust.

  • Robust: A system is robust when it can continue functioning in the presence of internal and external challenges without fundamental changes to the original system.

  • Resilient: A system is resilient when it can adapt to internal and external challenges by changing its method of operations while continuing to function. While elements of the original system are present there is a fundamental shift in core activities that reflects adapting to the new environment.

I think that it is a better idea to think about this from a system perspective rather than the specific X-risks or plans that we know about or think are cool. We want to avoid the availability bias. I would assume that there are more X-risks and plans that we are unaware of then we are aware of.

I recommend adding in the risks and relating them to the plans as most of your plans if they fail would lead to other risks. I would do this in a generic way. An example to demonstrate what I am talking about is: with a risk tragedy of the commons and a plan to create a more capable type of intelligent life form that will uphold, improve and maintain the interests of humanity. This could be done by genetic engineering and AI to create new life forms. And, Nanotechnology and biotechnology could be used to change existing humans. The potential risk of this plan is that it leads to the creation of other intelligent species that will inevitably compete with humans.

One more recommendation is to remove the time line from the road map and just have the risks and plans. The timeline would be useful in the explanation text you are creating. I like this categorisation of X risks:

  • Bangs (extinction) – Earth-originating intelligent life goes extinct in relatively sudden disaster resulting from either an accident or a deliberate act of destruction.

  • Crunches (permanent stagnation) – The potential of humankind to develop into posthumanity is permanently thwarted although human life continues in some form.

  • Shrieks (flawed realization) – Some form of posthumanity is attained but it is an extremely narrow band of what is possible and desirable.

  • Whimpers(subsequent ruination) – A posthuman civilization arises but evolves in a direction that leads gradually but irrevocably to either the complete disappearance of the things we value or to a state where those things are realized to only a minuscule degree of what could have been achieved.

I don’t want this post to be too long, so I have just listed the common systems problems below:

  • Policy Resistance – Fixes that Fail

  • Tragedy of the Commons

  • Drift to Low Performance

  • Escalation

  • Success to the Successful

  • Shifting the Burden to the Intervenor—Addiction

  • Rule Beating

  • Seeking the Wrong Goal

  • Limits to Growth

Four additional plans are:

  1. (in Controlled regression) voluntary or forced devolution

  2. uploading human consciousness into a super computer

  3. some movement or event that will cause a paradigmatic change so that humanity becomes more existentially-risk aware

  4. dramatic societal changes to avoid some existential risks like the over use of resources. An example of this is in the book: The world inside.

You talk about being saved by non-human intelligence, but it is also possible that SETI could actually cause hostile aliens to find us. A potential plan might be to stop SETI and try to hide. The opposite plan (seeking out aliens) seems as plausible though.

Comment author: Romashka 12 June 2015 04:36:43PM 1 point [-]

This is useful. Mr. Turchin, please redirect my award to Satoshi.

Comment author: turchin 13 June 2015 06:52:02PM 0 points [-]

Done