KatjaGrace comments on Superintelligence 23: Coherent extrapolated volition - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (97)
Would it be so bad to lock in our current values? (e.g. Compared to the other plausible dangers inherent in a transition to AI?)
I might not mind locking in my current values, but I sure don't want to lock in your current values.
No, more serious: Yes, it would be bad. As I wrote in "The human problem",
Consider that homo sapiens is in some sense a creation of our primate ancestors. Would it have been bad if those primate ancestors had managed to put limits on the evolution that lead to homo sapiens such that the evolution could never create a species which might supplant the ancestor? That would successfully force the any evolved knew species to serve the old one, perhaps to use its technology to cure the diseases of the primate ancestor, to tile the world in bananas and coconuts, to devote itself to the preservation and enhancement of primate ancestor culture?
I guess it would be bad for homo sapiens, but not so bad for the primate ancestors?
Would our primate ancestors be open to a charge that they were creating a race of slaves by limiting their evolution thusly? Of course, it seems more than likely that creating a race of slaves would not be ruled out by our primate ancestors CEV.