RichardKennaway comments on Superintelligence 9: The orthogonality of intelligence and goals - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (78)
How do you propose to discover the utility function of an agent by observing its actions? You will only ever see a tiny proportion of the possible situations it could be in, and in those situations you will not observe any of the actions it could have made but did not.
Observationally, you can't. But given its source code...