Emile comments on Extraterrestrial paperclip maximizers - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (157)
The creation of a powerful optimization process is a distraction here - as Eliezer points out in the article you link, and in others like the "Three Worlds Collide" story, aliens are quite unlikely to share much of our value system.
Yes, the creation of unfriendly AI is an important topic here on earth, but for an alien civilization, all we need to know is that they're starting off with a different values system, and that they might become very powerful. Meeting a powerful entity with a different value system is more likely to be bad news than good news, regardless of whether the "power" comes from creating an "alien-friendly" AI, an "alien-unfriendly" AI (destroying their old alien value system in the process), self modification, uploading or whatnot.
Berating Steven Hawking for not mentioning this detail ("Maybe the aliens will create an unfriendly AI!") is unnecessary. His view looks much less naive than this: