Mitchell_Porter comments on Open Thread: March 2010, part 2 - Less Wrong

4 Post author: RobinZ 11 March 2010 05:25PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (334)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 11 March 2010 06:06:02PM 2 points [-]

One career path I'm sort of musing about is working to create military robots. After all, the goals in designing a military robot are similar to those in designing Friendly AI: the robot must know somehow who it's okay to harm and what "harm" is.

Does this seem like a good sort of career path for someone interested in Friendly AI?

Comment author: Mitchell_Porter 17 March 2010 07:48:25AM 0 points [-]

Cognitive neuroscience and cognitive psychology are far more relevant. A Friendly AI is a moral agent; it's more like a judge than a cruise missile. A killer robot must inflict harm appropriately but it does not need to know what "harm" is; that's for politicians, generals, and other strategists.

We have to extract the part of the human cognitive algorithm which, on reflection, encodes the essence of rational and moral judgment and action. That's the sort of achievement which FAI will require.