FinalState comments on Holden's Objection 1: Friendliness is dangerous - Less Wrong

11 Post author: PhilGoetz 18 May 2012 12:48AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (428)

You are viewing a single comment's thread. Show more comments above.

Comment author: FinalState 23 May 2012 08:53:16PM -2 points [-]

ohhhh... sorry... There is really only one, and everything else is derived from it. Familiarity. Any other values would depend on the input, output and parameters. However familiarity is inconsistent with the act of killing familiar things. The concern comes in when something else causes the instance to lose access to something it is familiar with, and the instance decides it can just force that to not happen.

Comment author: TimS 23 May 2012 09:06:02PM 1 point [-]

Well, I'm not sure that Familiarity is sufficient to resolve every choice faced by a GIA - for example, how does one derive a reasonable definition of self-defense from Familiarity. But let's leave that aside for a moment.

Why must a GIA subscribe to the value of Familiarity?

Comment author: gwern 23 May 2012 09:12:16PM 0 points [-]

However familiarity is inconsistent with the act of killing familiar things.

"I Have No Mouth, and I Must Scream".