Wei_Dai comments on Evaluating the feasibility of SI's plan - Less Wrong

25 Post author: JoshuaFox 10 January 2013 08:17AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (186)
Tags: agi friendliness singularity institute si strategy ai safety

Comments (186)

You are viewing a single comment's thread. Show more comments above.

Comment author: Wei_Dai 11 January 2013 10:35:55PM 2 points [-]

where it leaves the Earth alone and so we get a lower bound on how disastrous a mistake can be

I don't see how to make this work. Do we make the AI indifferent about Earth? If so, Earth will be destroyed as a side effect of its other actions. Do we make it block all causal interactions between Earth and the rest of the universe? Then we'll be permanently stuck on Earth even if the FAI attempt turns out to be successful in other regards. Any other ideas?

Comment author: gwern 11 January 2013 10:52:23PM 0 points [-]

Do we make the AI indifferent about Earth? If so, Earth will be destroyed as a side effect of its other actions.

I had a similar qualm about the suicide button

Do we make it block all causal interactions between Earth and the rest of the universe? Then we'll be permanently stuck on Earth even if the FAI attempt turns out to be successful in other regards.

Nothing comes for free.

Powered by Reddit Powered by Reddit