You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

diegocaleiro comments on Superintelligence 11: The treacherous turn - Less Wrong Discussion

10 Post author: KatjaGrace 25 November 2014 02:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (50)

You are viewing a single comment's thread. Show more comments above.

Comment author: diegocaleiro 25 November 2014 08:13:42PM 1 point [-]

One alternative possibility is that the AI's utility function will converge somewhat slowly as it's capabilities increase and it's intelligence increase. While it has not converged yet, it would behave somewhat non-perfectly, and we would consider it's behaviors to be the stochastic precedents to a convergent phase: in other words, noise.

It would then have an incentive not necessarily to conceal the general direction towards which it is headed, but instead the noise rate of it's normal responses. It could pretend to be an ethically clumsy automaton, like Koba the chimp does in Dawn of the Planet of the Apes in the scene in which he steals armament from humans.... not without killing them first.