Nornagest comments on Causal Universes - Less Wrong

60 Post author: Eliezer_Yudkowsky 29 November 2012 04:08AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (385)

You are viewing a single comment's thread. Show more comments above.

Comment author: Nornagest 30 November 2012 03:01:59AM *  1 point [-]

I guess we're looking at interpretation 2, then. The main problem I see with that is that for most sapient systems, it's possible to imagine a subsapient system capable of organizing itself into a similar class of being, and it doesn't seem especially consistent for a set of morals to prohibit creating the former outright and remain silent on the latter.

Imagine for example a sapient missile guidance system. Your moral framework seems to prohibit creating such a thing outright, which I can see reasoning for -- but it doesn't seem to prohibit creating a slightly nerfed version of the same software that predictably becomes sapient once certain criteria are met. If you'd say that's tantamount to creating a sapient being, then fine -- but I don't see any obvious difference in kind between that and creating a human child, aside from predicted use.

Comment author: Decius 30 November 2012 03:43:08PM 0 points [-]

What's wrong with creating a sapient missile guidance system? What's the advantage of a sapient guidance system over a mere computer?

Given the existence of a sapient missile, it becomes impermissible to launch that missile without the consent of the missile. Just like it is impermissible to launch a spaceship without the permission of a human pilot...