Luke_A_Somers comments on Superintelligence 23: Coherent extrapolated volition - Less Wrong

5 Post author: KatjaGrace 17 February 2015 02:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (97)

You are viewing a single comment's thread. Show more comments above.

Comment author: Luke_A_Somers 17 February 2015 05:00:12PM *  1 point [-]

We never bother running a computer program unless we don't know the output and we know an important fact about the output. -- Marcello Herreshoff

In this case, one of the important facts must be that it won't go around changing its motivational structure. If it isn't, we're screwed for the reason you give.