Nick_Tarleton comments on True Ending: Sacrificial Fire (7/8) - Less Wrong

19 Post author: Eliezer_Yudkowsky 05 February 2009 10:57AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (90)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Nick_Tarleton 05 February 2009 06:05:34PM 1 point [-]

Also, I'm not at all confident that compromising with the Superhappies would be very bad, even before considering the probably larger benefit of them becoming more like us. I think I'd complain more about the abruptness and exogenousness of the change than the actual undesirability of the end state. As others have pointed out, though, a policy of compromise would lead to dilution of everyone's values into oblivion, and so may be highly undesirable.

More generally and importantly, though, I wonder if the use of wireheading as a standard example of "the hidden complexity of wishes" and of FAI philosophical failure (and it is an excellent example) leads me and/or other Singularitarians to have too negative a reaction to, well, anything that sounds like wireheading, including eliminating pain.