Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Grant comments on Can't Unbirth a Child - Less Wrong

24 Post author: Eliezer_Yudkowsky 28 December 2008 05:00PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (94)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Grant 29 December 2008 04:22:33AM 0 points [-]

Nick,

Understood; though I'd call fraud coercion, the use of the word is a side-issue here. However, an AI improving humans could have an equally clear view of what not to mess with: their current goal system. Indeed, I think if we saw specialized AIs that improved other AIs, we'd see something like this anyway. The improved AI would not agree to be altered unless doing so furthered its goals; i.e. the improving was unlikely to alter its goal system.