eli_sennesh comments on Why I haven't signed up for cryonics - Less Wrong

29 Post author: Swimmer963 12 January 2014 05:16AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (249)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 15 January 2014 05:53:44PM *  0 points [-]

I have, but it's running with the dramatic-but-unrealistic "genie model" of AI, in which you could simply command the machine, "Be a Friendly AI!" or "Be the CEV of humanity!", and it would do it. In real life, verbal descriptions are mere shorthand for actual mental structures, and porting the necessary mental structures for even the slightest act of direct normativity over from one mind-architecture to another is (I believe) actually harder than just using some form of indirect normativity.

(That doesn't mean any form of indirect normativity will work rightly, but it does mean that Evil Genie AI is a generalization from fictional evidence.)

Hence my saying I have trouble coming up with a realistic model.