Larks comments on The mathematics of reduced impact: help needed - Less Wrong

10 Post author: Stuart_Armstrong 16 February 2012 02:23PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (94)

You are viewing a single comment's thread. Show more comments above.

Comment author: Larks 17 March 2012 02:56:25AM 0 points [-]

use an L-infinity norm for deviations (across every moment of time as well).

The future 10^8 years later is going to look very different, even if things go right (FAI style or whatever), simply because we'll have used the AI for something. This is going to push your L-infinity norm very high, regardless of it's actions now, which is obviously very bad. As such, I think you want to weigh it be e^-t or something.

My other concern is that the AI will note that dedicating lots of resources to learning how to obey (game) the system will result in a really low score.