= 27d567bc2d48d9f40e9ccc20ea370b15)
srdiamond comments on Timeless Decision Theory: Problems I Can't Solve - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (153)
If you think being a rational agent includes an infinite ability to modify oneself, then the game has no solution because such an agent would be unable to guarantee the new trait's continued, unmodified existence without sacrificing the rationality that is a premise of the game.
So, for the game to be solvable, the self-modification ability must have limits, and the limits appear as parameters in the formalism.
An agent can guarantee the persistence of a trait by self-modifying into code that provably can never lead to the modification of that trait. A trivial example is that the agent can self-modify into code that preserves a trait and can't self-modify.
But more precisely, an agent can guarantee the persistence of a trait only "by self-modifying into code that provably can nevenrlead to the modification of that trait." Anything tied to rationality that guarantees the existence of a conforming modification at the time of offer must guarantee the continued existence of the same capacity after the modification, making the proposed self-modification self-contradictory.