Doug_S. comments on Ghosts in the Machine - Less Wrong

18 Post author: Eliezer_Yudkowsky 17 June 2008 11:29PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (27)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Doug_S. 18 June 2008 05:34:22AM 5 points [-]

Minimal, very imprecise description of Friendly AI:

"A Friendly AI is one that does what we ought to want it to do."

Unfortunately, we humans have enough trouble figuring out what we do want, let alone what we should want. Hence, even if we could make a Really Powerful Optimization Process that was better than people at achieving goals, we don't know what goals to make it try to achieve.