topynate comments on Precommitting to paying Omega. - Less Wrong

5 Post author: topynate 20 March 2009 04:33AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (27)

You are viewing a single comment's thread. Show more comments above.

Comment author: topynate 21 March 2009 10:31:43PM *  2 points [-]

the ideal-self-modifying-you

The ideal-potentially-self modifying me. No such being exists. I know, for a fact, that I am not perfectly rational in the sense that I construe "rational" to mean. That doesn't mean that Omega couldn't write a utility function that, if maximised, would perfectly describe my actions. Now in fact I am going to end up maximising that utility function: that's just mathematics/physics. But I am structured so as to value "me", even if "me" is just a concept I hold of myself. When I talk of ideal-Nathan, I mean a being that has the utility function that I think I have, which is not the same as the utility function that I do have. I then work out what ideal-Nathan does. If I find it does something that I know for a fact I do not want to do, then I'm simply mistaken about ideal-Nathan - I'm mistaken about my own utility function. That means that by considering the behaviour of ideal-Nathan (not looking so ideal now, is he?) I can occasionally discover something about myself. In this case I've discovered:

  • I don't care about my past selves nearly as much as I thought I did
  • I place a stronger premium on not modifying myself in such a way as to find killing pleasurable than I do on human life itself.