Manfred comments on Sneaky Strategies for TDT - Less Wrong

8 Post author: drnickbone 25 May 2012 04:13PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (22)

You are viewing a single comment's thread. Show more comments above.

Comment author: Manfred 26 May 2012 09:56:06PM *  5 points [-]

unless it resolves never to allow itself to be outperformed on any problem (in TDT über alles fashion).

This is not actually possible. You can always play the "I simulated you and put the money in the place you don't choose" game.

It seems a real strain to describe either of them as unfair to TDT.

From this side of the screen, this looks like a property of you, not the problems. If we replace the statement about "relative numbers" in the future (we were having to make assumptions about that anyhow, so let's just save time and stick in the assumptions), then problem 2 reads "I simulated the best decision theory by definition X and put the money in the place it doesn't choose." This demonstrates that no matter how good a decision theory is by any definition, it can still get hosed by Omega. In this case we're assuming that definition X is maximized by TDT (thus, it's a unique specification), and yea, TDT did go forth and get hosed by Omega.

Comment author: drnickbone 26 May 2012 10:41:57PM *  0 points [-]

This is not actually possible. You can always play the "I simulated you and put the money in the place you don't choose" game

But the obvious response to that game is randomisation among the choice options: there is no guarantee of winning, but no-one else can do better than you either. It takes a new "twist" on the problem to defeat the randomisation approach, and show that another agent type can do better.

I did ask on my original post (on Problematic Problems) whether that "twist" had been proposed or studied before. There were no references, but if you have one, please let me know.

Comment author: Manfred 26 May 2012 10:58:10PM 0 points [-]

I don't have such a reference - so good job :D And yes, I was assuming that Omega was defeating randomization.

Comment author: MugaSofer 29 December 2012 07:37:59PM -1 points [-]

It seems a real strain to describe either of them as unfair to TDT.

From this side of the screen, this looks like a property of you, not the problems. If we replace the statement about "relative numbers" in the future (we were having to make assumptions about that anyhow, so let's just save time and stick in the assumptions), then problem 2 reads "I simulated the best decision theory by definition X and put the money in the place it doesn't choose." This demonstrates that no matter how good a decision theory is by any definition, it can still get hosed by Omega. In this case we're assuming that definition X is maximized by TDT (thus, it's a unique specification), and yea, TDT did go forth and get hosed by Omega.

So there's a class of problems where failure is actually a good sign? Interesting. You might want to post further on that, actually.

Comment author: Manfred 29 December 2012 09:53:30PM 1 point [-]

Hm, yeah. After some computational work at least. Every decision procedure can get hosed by Omega, and the way in which it gets hosed is diagnostic of its properties. Though not uniquely, I guess, so you can't say "it fails this special test therefore it is good."