wedrifid comments on BOOK DRAFT: 'Ethics and Superintelligence' (part 1) - Less Wrong

11 Post author: lukeprog 13 February 2011 10:09AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (107)

You are viewing a single comment's thread. Show more comments above.

Comment author: wedrifid 15 February 2011 08:53:11AM *  5 points [-]

wedrifid is right: if you're now counting on failsafes to stop CEV from doing the wrong thing, that means you could apply the same procedures to any other proposed AI, so the real value of your life's work is in the failsafe, not in CEV.

Since my name was mentioned I had better confirm that I generally agree with your point but would have left out this sentence:

What happened to all your clever arguments saying you can't put external chains on an AI?

I don't disagree with the principle of having a failsafe - and don't think it is incompatible with the aforementioned clever arguments. But I do agree that "but there is a failsafe" is an utterly abysmal argument in favour of preferring CEV<humanity> over an alternative AI goal system.

I just don't understand this at all.

Tell me about it. With most people if they kept asking the same question when the answer is staring them in the face and then act oblivious as it is told to them repeatedly I dismiss them as either disingenuous or (possibly selectively) stupid in short order. But, to borrow wisdom from HP:MoR:

.... that just doesn't sound like /Eliezer's/ style.

...but you can only think that thought so many times, before you start to wonder about the trustworthiness of that whole 'style' concept.