You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Zetetic comments on What is/are the definition(s) of "Should"? - Less Wrong Discussion

4 Post author: Will_Sawin 01 June 2011 05:55PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (46)

You are viewing a single comment's thread. Show more comments above.

Comment author: Zetetic 01 June 2011 09:31:40PM 1 point [-]
  1. I suppose this is true. As long as your action is in conflict with my utility function, I will think that you "shouldn't" do it.

  2. I agree with that.

The triangulated "should-expression" in my above example is an expression of my utility function, but it is indirect insofar that it's a calculation given that your utility function conflicts substantially with mine.

Also, when I was talking about divergence before I realize that I was being sloppy. Our utility functions can "diverge" quite a bit without "conflicting", and the can "conflict" quite a bit without "diverging"; that is, our algorithms can both be "win the game of tic-tac-toe in front of you", and thus be exactly the same, but still be in perfect conflict. So sorry about that, that was just sloppy thinking altogether on my part.

Comment author: Will_Sawin 01 June 2011 10:03:30PM 1 point [-]

Then we agree, and just had some terminology problems.