LESSWRONG
LW

Comment Permalink
Oleander10

I voted No, but I don't think there is anything wrong with pushing the button. The "gods-eye" view of utility is simply wrong (or incoherent) and it's what causes most of the utility paradoxes. No one experiences not existing after pressing the button, so no harm done. 

[Edit] I should qualify that voting Yes on this question could potentially do harm. So if you're going to press the button, better to do it silently.

Reply
See in context
by lorepieri
1 min read
A
29

-2

The rules:

  1. Pressing the button will instantly terminate every single human being (including you).
  2. Pressing the button is completely anonymous. Nobody will know you did it, now or in the far future.
  3. The button cannot malfunction.
  4. The button does not have any secondary effects.
  5. The button is available for everyone to be pressed (not exactly the same physical button).
  6. Meta rule: There are no clever exploits available. This is not a riddle, the button does only what is written above.

Please do not to take the result too seriously. Inspired by AI X-Risk conversations.

I'm not aware of a poll functionality here, so I created the poll externally: https://strawpoll.com/3RnYXleKBye  Results are public and can be seen at the link without voting. Voting does not require any signup. 

Existential riskSurveysWorld Optimization
Frontpage

-2

[Thought Experiment] Given a button to terminate all humanity, would you press it?
New Answer
New Comment

2 Answers sorted by
top scoring

Richard_Kennaway

*

20

I see you have one positive vote already. Lizardman, efilist, or just someone mistaking their depression for the state of the world outside themselves?

I thought that (2-5) made the question too “nice” and made a similar poll.

“Would you release a superintelligence that you know will immediately destroy all life, before falling inert itself, leaving an utterly dead planet?”

“The superintelligence is not alive or conscious and its incomprehensible goals result in the destruction of everything whose atoms it has a use for as it proliferates over the face of the Earth, including all life everywhere on the planet, after which it falls inert. The process may take days or it may take years but there is nothing that anyone can do to stop or even slow down the destruction. There will be untold suffering during the process but at the end the Earth will be silent, devoid of life forever. Do you release it?”

ETA 2024 Aug 02 21:38 UTC+1: Yes 2, No 3.

[-]lorepieri10

Damn, we did not last even 24hrs...  

Thanks for the alternative poll.  One would think that with rules 2 and 5 out of the way it should be harder to say Yes. 

Reply
[-]Oleander10

I voted No, but I don't think there is anything wrong with pushing the button. The "gods-eye" view of utility is simply wrong (or incoherent) and it's what causes most of the utility paradoxes. No one experiences not existing after pressing the button, so no harm done. 

[Edit] I should qualify that voting Yes on this question could potentially do harm. So if you're going to press the button, better to do it silently.

Reply

lorepieri

10

Yes voter, if you can read this: why? It would be great to get an explanation (anon).

5 comments, sorted by
top scoring
Click to highlight new comments since:
[-]Seth Herd54

Some malcontent or any Negative Utilitarian that truly believes the hardline definition is going to press it. What's the point of this question? Things would go off the rails way faster than this if lots of people started to have control of RSI-capable AGI that could take over the world for their preferred future as well as probably wipe everyone out (although that's probably way easier than taking over the world without catastrophic damage).

Reply
[-]lorepieri10

How confident are you that someone is going to press it? If it's pressed: what's the frequency of someone pressing it? What can learn from it? Does any of the rules 2-5 play a crucial role in the decision to press it? 

(we are still alive so far!)

Reply
[-]Ustice42

I was more or less going to say the same thing. No, I wouldn’t press the button except in the most extremely bad scenarios I can imagine. As for how confident I am in that, I’m pretty tempted to say certain. Whether it is due to nihilistic glee, curiosity, clumsiness, or sheer stupidity, that button is going to be pressed. Now, there are scenarios that I can imagine that delay things for a human-significant amount of time.

Factors that I can think of right now that would expand the timeline:

  • ease of access to a doom button
  • cost to access a doom button
  • time of access for a doom button
  • intentionality verification requirements to press a doom button
  • societal and cultural significance of the doom button
  • scale of knowledge of
    • doom buttons
    • how to access doom buttons
    • how to operate doom buttons

I do live in Florida, so my estimates may be atypical.

I would estimate that for about 100 000 people the chance of someone pressing a doom button sitting right in front of them with full instructions on any given day would be around 1:100 for odds. A roll of a d100 sounds about right there. So that’s 1.00e-8 per person, per day. Using 8 billion as the world population comparing, the magnitudes of 1.00e-8 and 8.00e9, the population is going to swamp the odds of a doom button being pressed in a short time.

Reply
[-]Seth Herd23

Out of 8 billion people? Very near 100%. Of your handful of respondents? I have no idea, nor why I should care.

I'm asking you what we can learn from it, because you clearly have an idea. If you want to wait for the poll to fill before saying more, that's fine.

Reply
[-]whestler10

The only sensible reason I can imagine to push the button: a belief that p:doom from AI or something else is inevitable, and it would be better to remove all of humanity now and gamble on the possibility of a different intelligent form of life evolving in a few millenia. 

Reply
Moderation Log
More from lorepieri
View more
Curated and popular this week
A
25