ChristianKl comments on Open thread, Jul. 04 - Jul. 10, 2016 - Less Wrong

4 Post author: MrMind 04 July 2016 07:02AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (80)

Sort By: Best

You are viewing a single comment's thread. Show more comments above.

Comment author: ChristianKl 06 July 2016 09:51:03AM 0 points [-]

I have a problem understanding why a utility function would ever "stick" to an AI, to actually become something that it wants to keep pursuing.

I think that's one of MIRI's research problems. Designing an self-modifying AI that doesn't change it's utility function isn't trival.