You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

adamzerner comments on Can we decrease the risk of worse-than-death outcomes following brain preservation? - Less Wrong Discussion

8 Post author: Synaptic 21 February 2015 10:58PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (31)

You are viewing a single comment's thread.

Comment author: adamzerner 22 February 2015 05:14:37PM 2 points [-]

1) Do you think that an opt-out clause is a useful-in-principle way to address your concerns?

Yes. In principle, you should better achieve your outcomes if you have more precise instructions. In principle, this seems sort of obvious.

In practice, I see two problems:

1) If the agency misinterprets your instructions.

2) Ambiguous instructions could facilitate corruption. In the same way that ambiguous laws do.

I'm not sure whether the upsides (more precise instructions allow you to better achieve your outcome) outweigh the downsides (1 and 2). My intuition is that I'm ~65% sure that the upsides outweigh the downsides, but that doesn't reflect much thought.

Comment author: jlp 22 February 2015 06:24:12PM 3 points [-]

Practical problem #3: The agency successfully understands your intentions, and is willing to implement them, but not able to implement them.

For example, a fast intelligence explosion removes their capability of doing so before they can pull the plug. Or a change in their legal environment makes it illegal for them to pull the plug (and they aren't willing to put themselves at legal risk to do so).