In order to ensure that this post delivers what it promises, I have added the following content warnings:
Content Notes:
Pure Hypothetical Situation: The claim that perfect theoretical rationality doesn't exist is restricted to a purely hypothetical situation. No claim is being made that this applies to the real world. If you are only interested in how things apply to the real world, then you may be disappointed to find out that this is an exercise left to the reader.
Technicality Only Post: This post argues that perfectly theoretical rationality doesn't exist due to a technicality. If you were hoping for this post to deliver more, well, you'll probably be disappointed.
Contentious Definition: This post (roughly) defines perfect rationality as the ability to maximise utility. This is based on Wikipedia, which defines rational agents as an agent that: "always chooses to perform the action with the optimal expected outcome for itself from among all feasible actions".
We will define the number choosing game as follows. You name any single finite number x. You then gain x utility and the game then ends. You can only name a finite number, naming infinity is not allowed.
Clearly, the agent that names x+1 is more rational than the agent that names x (and behaves the same in every other situation). However, there does not exist a completely rational agent, because there does not exist a number that is higher than every other number. Instead, the agent who picks 1 is less rational than the agent who picks 2 who is less rational than the agent who picks 3 and so on until infinity. There exists an infinite series of increasingly rational agents, but no agent who is perfectly rational within this scenario.
Furthermore, this hypothetical doesn't take place in our universe, but in a hypothetical universe where we are all celestial beings with the ability to choose any number however large without any additional time or effort no matter how long it would take a human to say that number. Since this statement doesn't appear to have been clear enough (judging from the comments), we are explicitly considering a theoretical scenario and no claims are being made about how this might or might not carry over to the real world. In other words, I am claiming the the existence of perfect rationality does not follow purely from the laws of logic. If you are going to be difficult and argue that this isn't possible and that even hypothetical beings can only communicate a finite amount of information, we can imagine that there is a device that provides you with utility the longer that you speak and that the utility it provides you is exactly equal to the utility you lose by having to go to the effort to speak, so that overall you are indifferent to the required speaking time.
In the comments, MattG suggested that the issue was that this problem assumed unbounded utility. That's not quite the problem. Instead, we can imagine that you can name any number less than 100, but not 100 itself. Further, as above, saying a long number either doesn't cost you utility or you are compensated for it. Regardless of whether you name 99 or 99.9 or 99.9999999, you are still choosing a suboptimal decision. But if you never stop speaking, you don't receive any utility at all.
I'll admit that in our universe there is a perfectly rational option which balances speaking time against the utility you gain given that we only have a finite lifetime and that you want to try to avoid dying in the middle of speaking the number which would result in no utility gained. However, it is still notable that a perfectly rational being cannot exist within a hypothetical universe. How exactly this result applies to our universe isn't exactly clear, but that's the challenge I'll set for the comments. Are there any realistic scenarios where the lack of existence of perfect rationality has important practical applications?
Furthermore, there isn't an objective line between rational and irrational. You or I might consider someone who chose the number 2 to be stupid. Why not at least go for a million or a billion? But, such a person could have easily gained a billion, billion, billion utility. No matter how high a number they choose, they could have always gained much, much more without any difference in effort.
I'll finish by providing some examples of other games. I'll call the first game the Exploding Exponential Coin Game. We can imagine a game where you can choose to flip a coin any number of times. Initially you have 100 utility. Every time it comes up heads, your utility triples, but if it comes up tails, you lose all your utility. Furthermore, let's assume that this agent isn't going to raise the Pascal's Mugging objection. We can see that the agent's expected utility will increase the more times they flip the coin, but if they commit to flipping it unlimited times, they can't possibly gain any utility. Just as before, they have to pick a finite number of times to flip the coin, but again there is no objective justification for stopping at any particular point.
Another example, I'll call the Unlimited Swap game. At the start, one agent has an item worth 1 utility and another has an item worth 2 utility. At each step, the agent with the item worth 1 utility can choose to accept the situation and end the game or can swap items with the other player. If they choose to swap, then the player who now has the 1 utility item has an opportunity to make the same choice. In this game, waiting forever is actually an option. If your opponents all have finite patience, then this is the best option. However, there is a chance that your opponent has infinite patience too. In this case you'll both miss out on the 1 utility as you will wait forever. I suspect that an agent could do well by having a chance of waiting forever, but also a chance of stopping after a high finite number. Increasing this finite number will always make you do better, but again, there is no maximum waiting time.
(This seems like such an obvious result, I imagine that there's extensive discussion of it within the game theory literature somewhere. If anyone has a good paper that would be appreciated).
Link to part 2: Consequences of the Non-Existence of Rationality
You are right, theory is overrated. Just because you don't have a theoretical justification for commencing an action doesn't mean that the action isn't the right action to take if you want to try to "win." Of course, it is very possible to be in a situation where "winning" is inherently impossible, in which case you could still (rationally) attempt various strategies that seem likely to make you better off than you would otherwise be...
As a practicing attorney, I've frequently encountered real-life problems similar to the above. For example, in a negotiation on behalf of a client, there is often what's called a "bargaining zone" that represents a range of options for possible "deals" that both parties are theoretically willing to accept. Any given "deal" would be Pareto Efficient, and any "deal" within the "bargaining zone," if it takes place, would make both parties to the negotiation better off than they were before. However, it is possible to strike a superior deal for your client if you are more aggressive and push the terms into the "upper" range of the bargaining zone. On the other hand, you don't typically know the extent of the "bargaining zone" before you begin negotiations. If you are TOO aggressive and push outside of the range of the other party's acceptable options, the other party/counsel might get frustrated with you and call off the negotiations entirely, in which case you will lose the deal for everyone and make your client angry with you.
To the extent "winning" is possible here, the strategy for attorneys on both sides is to push the terms of the "deal" as close as possible to the "edge" of what the other will accept without pushing too far and getting the talks called off. Although there are reasonable strategies to the process, very often there isn't a theoretical "optimally rational strategy" for "winning" a negotiation -- you just have to play the game and make your strategic decisions based on new information as it becomes available.
There is an optimal strategy for negotiation. It requires estimating the negotiation zone of the other party and the utility of various outcomes (including failure of negotiation).
Then it's just a strategy that maximizes the sum of the probability of each outcome times the utility thereto.
The hard parts aren't the P(X1)U(X1) sums, it's getting the P(X1) and U(X1) in the first place.