eirenicon comments on The Strangest Thing An AI Could Tell You - Less Wrong

81 Post author: Eliezer_Yudkowsky 15 July 2009 02:27AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (574)

You are viewing a single comment's thread. Show more comments above.

Comment author: eirenicon 28 July 2009 03:06:08PM 0 points [-]

Would an AGI ever try to convince you of something you can't understand? I wouldn't try to explain special relativity to a kindergarten class. Surely an AGI would know perfectly well what you are capable of grasping. If it tries to convince me of something, knowing it cannot, what then are its intentions?

Comment author: UnholySmoke 29 July 2009 02:50:00PM 4 points [-]

Ack. 'Surely an AGI would be able to...' should be made illegal. I can quite easily conceive of an artificial mind that cannot model my thought processes. There's a great big long stretch of cleverness above human level before you reach omniscience!

There are also some humans who can understand lots of things, and some who can understand only very few things. If I'm being asked to sever a limb or stamp on a puppy, I at least want my shiny new master to have a stab at explaining why.