Stuart_Armstrong comments on Heroin model: AI "manipulates" "unmanipulatable" reward - All

6 Post author: Stuart_Armstrong 22 September 2016 10:27AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (10)

You are viewing a single comment's thread. Show more comments above.

Comment author: Stuart_Armstrong 26 September 2016 12:29:27PM 0 points [-]

Heroin might as well replace the human with a different entity, that has a slightly different utility function.

We feel that that is true, but "heroin replaces the human's utility" and "humans have composite utility where heroin is concerned" both lead to identical predictions. So you can't deduce the human's utility merely from observation; you need priors over what is irrational and what isn't.