casebash comments on The Number Choosing Game: Against the existence of perfect theoretical rationality - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (151)
Ok, lets say you are right that there does not exist perfect theoretical rationality in your hypothetical game context with all the assumptions that helps to keep the whole game standing. Nice. So what?
Spoilers, haha.
I was actually reading this post and I was trying to find a solution to the coalition problem where Eliezer wonders how rational agents can solve a problem with the potential for an infinite loop, which lead me to what I'll call the Waiting Game, where you can wait n units of time and gain n utility for any finite n, which then led me to this post.
Suppose instead that the game is "gain n utility". No need to speak the number, wait n turns, or even to wait for a meat brain to make a decision or comprehend the number.
I posit that a perfectly rational, disembodied agent would decide to select an n such that there exists no n higher. If there is a possible outcome that such an agent prefers over all other possible outcomes, then by the definition of utility such an n exists.
Not quite. There is no reason inherent in the definition that utility has to be bounded.