Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Eliezer_Yudkowsky comments on Taboo Your Words - Less Wrong

72 Post author: Eliezer_Yudkowsky 15 February 2008 10:53PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (128)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Eliezer_Yudkowsky 18 February 2008 06:26:56PM 4 points [-]

I'm not trying to under/over/middle-estimate you, only theories which you publicly write about. Sometimes I'm a real meanie with theories, shoving hot pokers into to them and all sorts of other nasty things. To me theories have no rights.

I know. But come on, you don't think the thought would ever have occurred to me, "I wonder if I can define Friendly AI without saying 'Friendly'?" It's not as if I invented the phrase first and only then thought to ask myself what it meant.

Moral, right, correct, wise, are all fine words for humans to use, but you have to break something down into ones and zeroes before it can be programmed. In a sense, the whole art of AGI is playing rationalist-Taboo with all words that refer to aspects of mind.

So are you saying that if at present you played a taboo game to communicate what "FAI" means to you, the effort would fail? I am interested in the intricacies of the taboo game including it's failure modes.

It has an obvious failure mode if you try to communicate something too difficult without requisite preliminaries, like calculus without algebra. Taboo isn't magic, it won't let you cross a gap of months in an hour.

I actually already have a meaning for FAI in my head. It seems different from the way other people try to describe it. It's more concrete but seems less virtuous. It's something along the lines of "obey me".

Really? That's your concept of how to steer the future of Earth-originating intelligent life? "Shut up and do what I say"? Would you want someone else to follow that strategy, say Archimedes of Syracuse, if the future fell into their hands?

So you play Taboo well, but you don't seem to see the difficulties that require a solution deeper than "obey me", and it's hard to explain an answer before explaining the question. Just like if you don't know about the game of Taboo, someone answers "Just build an AI to do the nice thing!"

You should consider looking for problems and failure modes in your own answer, rather than waiting for someone else to do it. What could go wrong if an AI obeyed you?

Comment author: christopherj 03 December 2013 03:37:12PM -1 points [-]

"Obey me" is actually a sane approach to creating FAI. It's clear and simple. The obedient AI can then be used to create a FAI, assuming the author wishes to do so and is able to communicate the concept of friendliness (both prerequisites for creating a FAI on purpose). Since the FAI needs to obey a friendliness criteria, it needs to have an obey capability built in anyways. The author just needs to make sure not to say something stupid, which once again is a necessity anyways.

Comment author: TheOtherDave 03 December 2013 05:58:36PM 0 points [-]

You seem to be expecting an obedient AI to understand "obey me" to mean "do only what I say"... e.g., you expect the AI not to interpret hand gestures, for example.

Is that right?
If so, how confident are you of that expectation?

Comment author: christopherj 05 December 2013 08:03:13AM 0 points [-]

I'd expect the "obey me" aspect to be "read signed messages from this file or from your input and do what it says" then making sure that the AI can't get the signing key and cut out the middleman. Definitely not something as simple to overwrite or fake as microphone or keyboard inputs. Also that way I don't say things by accident, although any command could still have unintended consequences.

Comment author: TheOtherDave 05 December 2013 03:27:10PM 0 points [-]

OK, thanks for clarifying that.
Do you expect the signed messages to be expressed in a natural human language?

Comment author: christopherj 05 December 2013 06:32:09PM 0 points [-]

Unfortunately, that would be impossible, unless you can make an AI that can understand natural language before it is ever run. And that would require having a proper theory of mind right from the start.

Comment author: TheAncientGeek 05 December 2013 06:38:13PM -1 points [-]

Hello? Seed .AI?

Comment author: TheOtherDave 05 December 2013 07:12:19PM 0 points [-]

OK. Thanks for clarifying your expectations.