JoshuaFox comments on Superintelligence 12: Malignant failure modes - Less Wrong

7 Post author: KatjaGrace 02 December 2014 02:02AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (50)

You are viewing a single comment's thread. Show more comments above.

Comment author: JoshuaFox 02 December 2014 06:56:24AM 3 points [-]

Of course, there are two kinds of perversity.

Perversity is "a deliberate desire to behave in an unreasonable or unacceptable way; contrariness."

Fictional genies seek out ways to trick the requester on purpose, just to prove a point about rash wishes. The other kind of perverse agent doesn't act contrarily for the sake of being contrary. They act exactly to achieve the goal and nothing else; it's just that they go against implicit goals as a side effect.