Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

PK comments on Taboo Your Words - Less Wrong

71 Post author: Eliezer_Yudkowsky 15 February 2008 10:53PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (128)

Sort By: Old

You are viewing a single comment's thread.

Comment author: PK 18 February 2008 04:45:21PM 3 points [-]

@Richard Hollerith: Skipping all the introductory stuff to the part which tries to define FAI(I think), I see two parts. Richard Hollerith said:

"This vast inquiry[of the AI] will ask not only what future the humans would create if the humans have the luxury of [a)] avoiding unfortunate circumstances that no serious sane human observer would want the humans to endure, but also [b)] what future would be created by whatever intelligent agents ("choosers") the humans would create for the purpose of creating the future if the humans had the luxury"

a) What's a "serious sane human observer"? Taboo the words and synonyms. What are "unfortunate circumstances" that s/he would like to avoid? Taboo...

b)What is "the future humans would chose for the purpose of creating the future"? In what way exactly would they "chose" it? Taboo... Good luck :-) --------------------------------------------- Eliezer Yudkowsky said: "Don't underestimate me so severely. You think I don't know how to define "Friendly" without using synonyms? Who do you think taught you the technique? Who do you think invented Friendly AI?"

I'm not trying to under/over/middle-estimate you, only theories which you publicly write about. Sometimes I'm a real meanie with theories, shoving hot pokers into to them and all sorts of other nasty things. To me theories have no rights.

"... I've covered some of the ladder I used to climb to Friendly AI. But not all of it. So I'm not going to try to explain FAI as yet; more territory left to go." So are you saying that if at present you played a taboo game to communicate what "FAI" means to you, the effort would fail? I am interested in the intricacies of the taboo game including it's failure modes.

"But you (PK) are currently applying the Taboo technique correctly, which is the preliminary path I followed at the analogous point in my own reasoning; and I'm interested in seeing you follow it as far as you can. Maybe you can invent the rest of the ladder on your own. You're doing well so far. Maybe you'll even reach a different but valid destination." I actually already have a meaning for FAI in my head. It seems different from the way other people try to describe it. It's more concrete but seems less virtuous. It's something along the lines of "obey me".

Comment author: Normal_Anomaly 27 November 2011 11:06:02PM 1 point [-]

I actually already have a meaning for FAI in my head. It seems different from the way other people try to describe it. It's more concrete but seems less virtuous. It's something along the lines of "obey me".

I suspect that you are joking. However, I would not create an AGI with the utility function "obey Normal_Anomaly".

Comment author: wedrifid 28 November 2011 05:23:34AM *  1 point [-]

I actually already have a meaning for FAI in my head. It seems different from the way other people try to describe it. It's more concrete but seems less virtuous. It's something along the lines of "obey me".

Your position isn't too unusual. That is, assuming you mean by "obey me" something like "obey what I would say to you if I was a whole heap better at understanding and satisfying my preferences, etc". Because actually just obeying me sounds dangerous for obvious reasons.

Is that similar or different to what you would consider friendly? (And does Friendly need to do exactly the above or merely close enough? ie. I expect an FAI<Eliezer> would be 'friendly enough' to me for me to call it an FAI. It's not that much different to what I would want after all. I mean, I'd probably get to live indefinitely at least.)