Richard_Loosemore comments on Debunking Fallacies in the Theory of AI Motivation - LessWrong

8 Post author: Richard_Loosemore 05 May 2015 02:46AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (343)

You are viewing a single comment's thread. Show more comments above.

Comment author: TheAncientGeek 14 May 2015 12:39:50PM *  2 points [-]

. The way that the DLI is defined, it borders on self-evidently true

ETA

I now see that what you have written subsequently to the OP is that DLI is almost, but not quite a description of rigid behaviour as a symptom (with the added ingredient that an AI can see the mistakenness of its behaviour):-

However, suppose there is no safe mode, and suppose that the AI also knows about its own design. For that reason, it knows that this situation has come about because (a) its programming is lousy, and (b) it has been hardwired to carry out that programming REGARDLESS of all this understanding that it has, about the lousy programming and the catastrophic consequences for the strawberries.Now, my "doctrine of logical infallibility" is just a shorthand phrase to describe a superintelligent AI in that position which really is hardwired to go ahead with the plan, UNDER THOSE CIRCUMSTANCES. That is all it means. It is not about the rigidity as such, it is about the fact that the AI knows it is being rigid, and knows how catastrophic the consequences will be.

HOWEVER, that doesn't entirely gel with what you wrote in the OP;-

One way to characterize this assumption is that the AI is supposed to be hardwired with a Doctrine of Logical Infallibility. The significance of the doctrine of logical infallibility is as follows. The AI can sometimes execute a reasoning process, then come to a conclusion and then, when it is faced with empirical evidence that its conclusion may be unsound, it is incapable of considering the hypothesis that its own reasoning engine may not have taken it to a sensible place. The system does not second guess its conclusions. This is not because second guessing is an impossible thing to implement, it is simply because people who speculate about future AGI systems take it as a given that an AGI would regard its own conclusions as sacrosanct.

Emph added. Doing dumb things because you think are correct, DLI v1, just isnt the same as realising their dumbness, but being tragically compelled to do them anyway...DLI2. (And Infallibility is a much more appropriate label for the origin idea....the second is more like inevitability)

Comment author: Richard_Loosemore 14 May 2015 02:10:27PM 2 points [-]

I responded before you edited and added extra thoughts .... [processing...]