Luke_A_Somers comments on Superintelligence 13: Capability control methods - Less Wrong

7 Post author: KatjaGrace 09 December 2014 02:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (44)

You are viewing a single comment's thread. Show more comments above.

Comment author: Luke_A_Somers 09 December 2014 09:12:20PM 2 points [-]

I see. I thought you meant take some other AI and apply breakpoints at incremental goal boundaries, and reset its goal system at that time.

I don't think this would work. Eventually it's going to reach a local maximum and have to sit down and think much harder to do valley-crossing (this will likely come at the end of a sequence of longer and longer optimization times finding this peak, so run-time won't be surprising). Then it is forced to do the long-term-make-life-easier-'evil-things'.