= 27d567bc2d48d9f40e9ccc20ea370b15)
paulfchristiano comments on Working hurts less than procrastinating, we fear the twinge of starting - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (138)
That exploit works against a hyperbolic discounter who today wants to work tomorrow, but tomorrow doesn't want to work today.
It doesn't work against Clippy's example of an exponential discounter who doesn't want to work today and knows that tomorrow he still won't want to work today, but still claims to want to work someday, even though he can't say when.
Our agent cannot reason from "I want to work someday" to "There exists a day in the finitely distant future when I will want to work". He is missing some kind of reverse induction axiom. We agree that there is something wrong with this agent's thinking.
But, I don't see how to exploit that flaw.
Interestingly, Peano arithmetic has the same "problem." This isn't directly relevant, but it does very strongly suggest that there is no possible way to exploit this flaw.
Suppose I take some program which looks really complicated to PA. In particular, the program runs indefinitely but PA can't prove it. Then for every particular amount of time, PA can prove that the program hasn't yet stopped. But there are models of PA where it is nevertheless true that "There exists a time at which the program has stopped." It is intuitively like having two sets of integers. The normal integers, obtained from 0 by adding 1 a finite number of times, and the really large integers, obtained from the halting time of your program by adding or subtracting 1 a finite number of times. There is no way to get from one to the other, because the really large integers are just that large.
If you use to ZFC instead, you encounter significantly less intuitive versions of this strange behavior.
In our case, this would be like believing in a hypothetical future time where you will do work, but which can never be accessed by letting the days pass one by one.