multifoliaterose comments on Friendly AI Research and Taskification - Less Wrong

22 Post author: multifoliaterose 14 December 2010 06:30AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (44)

You are viewing a single comment's thread. Show more comments above.

Comment author: Mitchell_Porter 15 December 2010 08:57:01AM 3 points [-]

I'm not convinced that researching the Friendly AI concept is a cost-effective way of reducing existential risk.

Researching Friendly AI is a way to reduce the risk of Unfriendly AI.

Personally, I find it very easy to outline a recipe for FAI.

  1. You use cognitive neuroscience to figure out human values and human metaethical thought.

  2. You automate human metaethical thought, apply that to the values determined at step 1, and thereby arrive at a human-relative moral/behavioral ideal.

  3. You design an open-ended AGI architecture, and use the ideal from step 2 to supply it with a goal.

To me, that defines a research program that's just full of highly concrete tasks waiting to be carried out. Some of them may be very difficult or abstract, but nothing leaves me feeling helpless, with no idea where to begin. However, you need to know something about ordinary AI (try this), and you need to have some idea of what a computational model of human decision making might look like (simple examples).

Comment author: multifoliaterose 15 December 2010 08:18:49PM 0 points [-]

Thanks for your interesting comment.

To me, that defines a research program that's just full of highly concrete tasks waiting to be carried out. Some of them may be very difficult or abstract, but nothing leaves me feeling helpless, with no idea where to begin.

I imagine that writing up a description of many such concrete tasks would help clarify the situation; I'd encourage you to do so if you have some time.

However, you need to know something about ordinary AI (try this), and you need to have some idea of what a computational model of human decision making might look like (simple examples).

Yes, I don't have subject matter knowledge. Thanks for the links.