Eliezer_Yudkowsky comments on The Strangest Thing An AI Could Tell You - Less Wrong

81 Post author: Eliezer_Yudkowsky 15 July 2009 02:27AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (574)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 16 July 2009 04:35:32AM 2 points [-]

What does that mean, exactly?

Comment author: Alan 16 July 2009 03:01:16PM 3 points [-]

In reply, at a superficial level, the statement was intended as (wry) humor toward consequentialist friends in the community. Anyone who wrote the AI code presumably had a hypothetical imperative in mind: "You, the AI, must do such and such in order to reach specified ends, in this case reporting a truthful statement." And that's what AI does, right? But If the AI reports that deontology is the way to go and tells you that you owe AI reciprocal respect as a rational being bound by a certain priori duties and prohibitions, that sounds quite crazy--after all, it's only code. Yet might our ready to hand conceptions of law and freedom predispose us to believe the statement? Should we believe it?