Technologos comments on Reason as memetic immune disorder - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (166)
And not just by persuading the guards--the kind of AIs we are talking about, transhuman-level AIs, could potentially do all kinds of mind-hacking things of which we haven't even yet conceived. Hell, they could do things that we will never be able to conceive unaided.
If we ever set up a system that relies on humans restraining a self-modifying AI, we had better be sure beforehand that the AI is Friendly. The only restraints that I can think of that would provably work involve limiting the AIs access to resources so that it never achieves a level of intelligence equal to or higher than human--but then, we haven't quite made an AI, have we? Not much benefit to a glorified expert system.
If you haven't read the AI Box experiment reports I linked above, I recommend them--apparently, it doesn't quite take a transhuman-level AI to get out of a "test harness."