A putative new idea for AI control; index here.

This is one of the more minor suggestions, just a small tweak to help solve a specific issue.

Discounting time

The issue is the strange behaviour that agents with discount rates have in respect to time.

Quickly, what probability would you put on time travel being possible?

I hope, as good Bayesians, you didn't answer 0% (those who did should look here). Let's assume, for argument sake, that you answered 0.1%.

Now assume that you have a discount rate of 10% per year (many putative agent designs use discount rates for convergence or to ensure short time-horizons, where they can have discount rates of 90% per second or even more extreme). By the end of 70 years, the utility will be discounted to roughly 0.1%. Thus, from then on (plus or minus a few years), the highest expected value action for you is to search for ways of travelling back in time, and do all you stuff then.

This is perfectly time-consistent: given these premisses, you'd want the "you in a century" to search frantically for a time-machine, as the actual expected utility increase they could achieve is tiny.

If you were incautions enough to have discount rates that go back into the past as well as the future, then you'd already be searching frantically for a time-machine, for the tiniest change of going back to the big bang and having an impact there...