Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

JulianMorrison comments on That Alien Message - Less Wrong

111 Post author: Eliezer_Yudkowsky 22 May 2008 05:55AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (164)

Sort By: Old

You are viewing a single comment's thread.

Comment author: JulianMorrison 23 May 2008 08:53:00AM 0 points [-]

Eliezer: why would it be immoral to build a FAI as a "person"? To rewire a human as Friendly (to dumb aliens) would be immoral because it rewires their goals in a way the original goals would hate. However an AI which comes out of the compiler with Friendly goals would not view being Friendly as a rewire but as its ground state of existence. You seem very confident it's immoral, so I'm assuming you have a good reason. Please tell.

Comment author: pnrjulius 09 April 2012 05:04:44AM 0 points [-]

Also, if there really is an objective moral truth, then it is obligatory to rewire yourself into seeking it. (Whether or not you are willing to do so is a different question; but you ought to be.)