9eB1 comments on Circular belief updating - LessWrong

6 Post author: irrational 11 December 2013 06:26AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (50)

You are viewing a single comment's thread. Show more comments above.

Comment author: 9eB1 11 December 2013 07:30:49PM 0 points [-]

Sorcerers could influence people's minds to just change their goals/utility functions without stopping them from being perfect reasoners. This is why people worry so much about AI friendliness, because assuming infinite computational capacity there is still (we assume) no privileged utility function.