RichardKennaway comments on The Strangest Thing An AI Could Tell You
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You'll need to login or register to do that
(Don't worry, it only takes a few seconds)
Create a new account
All it takes is a username and password
Comments (278)
If the AGI can't convince me of something, maybe it's not because it's not smart enough to explain, but because I'm not smart enough to understand.
Would an AGI ever try to convince you of something you can't understand? I wouldn't try to explain special relativity to a kindergarten class. Surely an AGI would know perfectly well what you are capable of grasping. If it tries to convince me of something, knowing it cannot, what then are its intentions?
Ack. 'Surely an AGI would be able to...' should be made illegal. I can quite easily conceive of an artificial mind that cannot model my thought processes. There's a great big long stretch of cleverness above human level before you reach omniscience!
There are also some humans who can understand lots of things, and some who can understand only very few things. If I'm being asked to sever a limb or stamp on a puppy, I at least want my shiny new master to have a stab at explaining why.
Dead right. It would seem very silly to believe that rationality hits a glass ceiling at human level intelligence. Unlikely though it is, if the AI could predict the number in my head by looking at my facial expressions, then told me to cut my arm off for the good of the human race, I'd suddenly feel very conflicted indeed.