Dave5 comments on Ethics Notes - Less Wrong

12 Post author: Eliezer_Yudkowsky 21 October 2008 09:57PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (44)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Dave5 21 October 2008 11:27:05PM 1 point [-]

Because I would give odds around as extreme as the odds I would give of anything, that if you tell me "the AI you built is trying to deceive yourself", it indicates that some kind of really epic error has occurred. Controlled shutdown, immediately.

Um, no. Controlled shutdown means you are relying on software, which should be presumed corrupted, unless you are very sure about your correctness proofs. What you want there is uncontrolled shutdown, whether by pulling the plug, taking an axe to the CPU, shutting down the local power-grid, or nuking the city, as necessary. Otherwise, Hard Rapture.