All of wheninrome15's Comments + Replies

Is there any chance that we (a) CAN'T restrict AI to be friendly per se, but (b) (conditional on this impossibility) CAN restrict it to keep it from blowing up in our faces? If friendly AI is in fact not possible, then first generation AI may recognize this fact and not want to build a successor that would destroy the first generation AI in an act of unfriendliness.

It seems to me like the worst case would be that Friendly AI is in fact possible...but that we aren't the first to discover it. In which case AI would happily perpetuate itself. But what are ... (read more)

2PhilGoetz
First, define "friendly" in enough detail that I know that it's different from "will not blow up in our faces".
1RobinZ
Such an eventuality would seem to require that (a) human beings are not computable or (b) human beings are not Friendly. In the latter case, if nothing else, there is [individual]-Friendliness to consider.