x

LESSWRONG

LW

S-risks: Why they are the worst existential risks, and how to prevent them — LessWrong

Risks of Astronomical Suffering (S-risks)Cause PrioritizationSufferingWorld Optimization

31

S-risks: Why they are the worst existential risks, and how to prevent them

20th Jun 2017

1 min read

31

This is a linkpost for https://foundational-research.org/s-risks-talk-eag-boston-2017/

Risks of Astronomical Suffering (S-risks)Cause PrioritizationSufferingWorld Optimization

31

New Comment

Rendering 0/106 comments, sorted by

(show more) Click to highlight new comments since: Today at 12:35 PM

Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

More from Kaj_Sotala

Curated and popular this week

106

Comment Permalink

Any superintelligence that achieves human values will be adjacent in design space to many superintelligences that cause massive suffering

Why so? Flipping the sign doesn't get you "adjacent", it gets you "diametrically opposed".

If you really want chocolate ice cream, "adjacent" would be getting strawberry ice cream, not having ghost pepper extract poured into your mouth.

Good_Burning_Plastic9y30

They said "adjacent in design space". The Levenshtein distance between return val; and return -val; is 1.

0cousin_it9y

That's a twist on a standard LW argument, see e.g. here: [...] It seems to me that fragility of value can lead to massive suffering in many ways.

Mentioned in

35Bi-Weekly Rational Feed

15[Book Review] "Suffering-focused Ethics" by Magnus Vinding