Mitchell_Porter comments on A Master-Slave Model of Human Preferences - Less Wrong

58 Post author: Wei_Dai 29 December 2009 01:02AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (80)

You are viewing a single comment's thread. Show more comments above.

Comment author: Wei_Dai 29 December 2009 08:51:26PM 4 points [-]

I agree this is part of the problem, but like others here I think you might be making it out to be harder than it is. We know, in principle, how to translate a utility function into a physical description of an object: by coding it as an AI and then specifying the AI along with its substrate down to the quantum level. So, again in principle, we can go backwards: take a physical description of an object, consider all possible implementations of all possible utility functions, and see if any of them matches the object.

Comment author: Mitchell_Porter 30 December 2009 12:18:56PM 1 point [-]

Here's another statement of the problem: One agent's bias is another agent's heuristic. And the "two agents" might be physically the same, but just interpreted differently.