= 27d567bc2d48d9f40e9ccc20ea370b15)
TheOtherDave comments on Don't plan for the future - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (50)
As I understand the usage elsewhere on this site, a Friendly AI created by nonhumans ought to embody the terminal values of the creating race, just as we talk about FAIs created by humans embodying the terminal values of humans.
And presumably the same reasoning that concludes that a self-optimizing AI, unless created taking exquisite care to ensure Friendliness, won't actually be compatible with human values (due to the Vastness of mindspace and so forth), also concludes that a powerful alien race, having necessarily been created without taking such care, will similarly not be compatible with human values.
This line of reasoning seems to conclude that drawing a distinction between alien UFAIs and alien FAIs (and, for that matter, alien NIs) is moot in this context -- they are all a threat to humanity.
Which, yes, leads to exactly the same default assumption you cite.
There is some hope 1. ) the absence of any paperclippers out there is evidence that the idea of AI going FOOM is bogus 2.) our FAI will make a deal with other FAI's.
I agree that friendliness is subjective. CEV of humanity will equal paperclipping for most minds and even be disregarded by some human minds.