topynate comments on Precommitting to paying Omega. - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (27)
The ideal-potentially-self modifying me. No such being exists. I know, for a fact, that I am not perfectly rational in the sense that I construe "rational" to mean. That doesn't mean that Omega couldn't write a utility function that, if maximised, would perfectly describe my actions. Now in fact I am going to end up maximising that utility function: that's just mathematics/physics. But I am structured so as to value "me", even if "me" is just a concept I hold of myself. When I talk of ideal-Nathan, I mean a being that has the utility function that I think I have, which is not the same as the utility function that I do have. I then work out what ideal-Nathan does. If I find it does something that I know for a fact I do not want to do, then I'm simply mistaken about ideal-Nathan - I'm mistaken about my own utility function. That means that by considering the behaviour of ideal-Nathan (not looking so ideal now, is he?) I can occasionally discover something about myself. In this case I've discovered: