Epictetus comments on Debunking Fallacies in the Theory of AI Motivation - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (343)
The point doesn't need to be argued for on the basis of definitions. Given one set of assumptions, one systems architecture, it is entirely natural that an AI would pursue its goals against is own information, and against the protests of humans;. But on other assumptions, it is utterly bizarre that an AI would ever do that....it would be not merely an error, in the sense of a bug, a failure on the part of the programmers to code their intentions, but an unlikely kind of bug that allows the system to continue doing really complex things, instead of degrading it.
If one of its parameters is "do not go against human protests of magnitude greater than X", then it will not pursue a course of action if enough people protest it. But in this case, avoiding strong human protest is part of its goals.
The AI is ultimately following some procedure, and any outside information or programmer intention or human protest is just some variable that may or may not be taken into consideration.
That just restated my point that the different sides in the debate are just making different assumptions about likely AI architectures.
But the AI researchers win, because they know what real world AI architectures are, whereas MIRI is guessing.