komponisto comments on A Thought on Pascal's Mugging - Less Wrong

12 Post author: komponisto 10 December 2010 06:08AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (159)

You are viewing a single comment's thread. Show more comments above.

Comment author: komponisto 11 December 2010 06:39:43AM 1 point [-]

However, the utility function still isn't up for grabs. If our actual true utility function violates this rule, I don't want to say that an AGI is unfriendly for maximizing it.

Of course. The proposal here is that "our actual true utility function" does not violate this rule, since we are not in fact inclined to give in to a Pascalian mugger.

Comment author: JGWeissman 11 December 2010 07:21:03AM 0 points [-]

"our actual true utility function"

Sometimes, when I go back and read my own comments, I wonder just what goes on in that part of my brain that translates concepts into typed out words when I am not paying it conscious attention.

Anyways, let "our actual true utility function" refer to the utility function that best describes our collective values that we only manage to effectively achieve in certain environments that match the assumptions inherent in our heuristics. Thinking of it this way, one might wonder if Pascalian muggers fit into these environments, and if not, how much does our instinctual reaction to them indicate about our values?