= 27d567bc2d48d9f40e9ccc20ea370b15)
Nebu comments on The Number Choosing Game: Against the existence of perfect theoretical rationality - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (151)
Regardless of what number you choose, there will be another agent who chooses a higher number than you and hence who does better at the task of utility optimising than you do. If "perfectly rational" means perfect at optimising utility (which is how it is very commonly used), then such a perfect agent does not exist. I can see the argument for lowing the standards of "perfect" to something achievable, but lowering it to a finite number would result in agents being able to outperform a "perfect" agent, which would be equally confusing.
Perhaps the solution is to taboo the word "rational". It seems like you agree that there does not exist an agent that scores maximally. People often talk about utility-maximising agents, which assumes it is possible to have an agent which maximises utility, which isn't true for some situations. That the assumption I am trying to challenge regardless of whether we label it perfect rationality or something else.
Let's taboo "perfect", and "utility" as well. As I see it, you are looking for an agent who is capable of choosing The Highest Number. This number does not exist. Therefore it can not be chosen. Therefore this agent can not exist. Because numbers are infinite. Infinity paradox is all I see.
Alternately, letting "utility" back in, in a universe of finite time, matter, and energy, there does exist a maximum finite utility which is the sum total of the time, matter, and energy in the universe. There will be an number which corresponds to this. Your opponent can choose a number higher than this but he will find the utility he seeks does not exist.
Why can't my utility function be:
?
I.e. why should we forbid a utility function that returns infinity for certain scenarios, except insofar that it may lead to the types of problems that the OP is worrying about?
I was bringing the example into the presumed finite universe in which we live, where Maximum Utility = The Entire Universe. If we are discussing a finite-quantity problem than infinite quantity is ipso facto ruled out.
I think Nebu was making the point that while we normally use utility to talk about a kind of abstract gain, computers can be programmed with an arbitrary utility function. We would generally put certain restraints on it so that the computer/robot would behave consistently, but those are the only limitation. So even if there does not exist such a thing as infinite utility, a rational agent may still be required to solve for these scenarios.
I guess I'm asking "Why would a finite-universe necessarily dictate a finite utility score?"
In other words, why can't my utility function be: