JoshuaFox comments on Superintelligence 11: The treacherous turn - Less Wrong

10 Post author: KatjaGrace 25 November 2014 02:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (50)

You are viewing a single comment's thread. Show more comments above.

Comment author: JoshuaFox 25 November 2014 08:01:35AM 9 points [-]

To play the treacherous turn gambit, the AI needs to get strong at faking weakness faster than it gets strong at everything else. What are the chances of that?

Comment author: SteveG 28 November 2014 03:38:59AM 4 points [-]

I hear you and kind of agree. On the other hand, when a 3-year old lies, sometimes they to pull it off.

Comment author: William_S 29 November 2014 02:29:00AM *  1 point [-]

I feel like the chances of a treacherous turn happening with no warning at all are unlikely unless the intelligence rise is very rapid. However, it also seems that past a certain point, the treachery will be very hard to detect. Risk may not be so much that there are no warning signs, but that the warning signs are ignored.