You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Nebu comments on The Number Choosing Game: Against the existence of perfect theoretical rationality - Less Wrong Discussion

-1 Post author: casebash 29 January 2016 01:04AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (151)

Comments (151)

You are viewing a single comment's thread. Show more comments above.

Comment author: Usul 05 January 2016 05:28:57AM *  3 points [-]

Let's taboo "perfect", and "utility" as well. As I see it, you are looking for an agent who is capable of choosing The Highest Number. This number does not exist. Therefore it can not be chosen. Therefore this agent can not exist. Because numbers are infinite. Infinity paradox is all I see.

Alternately, letting "utility" back in, in a universe of finite time, matter, and energy, there does exist a maximum finite utility which is the sum total of the time, matter, and energy in the universe. There will be an number which corresponds to this. Your opponent can choose a number higher than this but he will find the utility he seeks does not exist.

Comment author: Nebu 05 January 2016 07:56:46AM 1 point [-]

Alternately, letting "utility" back in, in a universe of finite time, matter, and energy, there does exist a maximum finite utility which is the sum total of the time, matter, and energy in the universe.

Why can't my utility function be:

  • 0 if I don't get ice cream
  • 1 if I get vanilla ice cream
  • infinity if I get chocolate ice cream

?

I.e. why should we forbid a utility function that returns infinity for certain scenarios, except insofar that it may lead to the types of problems that the OP is worrying about?

Comment author: Usul 05 January 2016 08:07:09AM 0 points [-]

I was bringing the example into the presumed finite universe in which we live, where Maximum Utility = The Entire Universe. If we are discussing a finite-quantity problem than infinite quantity is ipso facto ruled out.

Comment author: casebash 05 January 2016 11:52:44PM 1 point [-]

I think Nebu was making the point that while we normally use utility to talk about a kind of abstract gain, computers can be programmed with an arbitrary utility function. We would generally put certain restraints on it so that the computer/robot would behave consistently, but those are the only limitation. So even if there does not exist such a thing as infinite utility, a rational agent may still be required to solve for these scenarios.

Comment author: Nebu 24 January 2016 10:44:33PM *  0 points [-]

I guess I'm asking "Why would a finite-universe necessarily dictate a finite utility score?"

In other words, why can't my utility function be:

  • 0 if you give me the entire universe minus all the ice cream.
  • 1 if you give me the entire universe minus all the chocolate ice cream.
  • infinity if I get chocolate ice cream, regardless of how much chocolate ice cream I receive, and regardless of whether the rest of the universe is included with it.
Powered by Reddit Powered by Reddit