Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

# Jonii comments on Problematic Problems for TDT - LessWrong

34 29 May 2012 03:41PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Sort By: Best

You are viewing a single comment's thread.

Comment author: 23 May 2012 09:32:40PM 0 points [-]

Interaction of this simulated TDT and you is so complicated I don't think many of commenters here actually did the math to see how should they expect the simulated TDT agent to react in these situations. I know I didn't. I tried, and failed.

Comment author: 24 May 2012 09:35:39AM *  3 points [-]

Maybe I'm missing something, but the formalization looks easy enough to me...

def tdt_utility():
if tdt(tdt_utility) == 1:
box1 = 1000
box2 = 1000000
else:
box1 = 1000
box2 = 0
if tdt(tdt_utility) == 1:
return box2
else:
return box1+box2
def your_utility():
if tdt(tdt_utility) == 1:
box1 = 1000
box2 = 1000000
else:
box1 = 1000
box2 = 0
if you(your_utility) == 1:
return box2
else:
return box1+box2

The functions tdt() and you() accept the source code of a function as an argument, and try to maximize its return value. The implementation of tdt() could be any of our formalizations that enumerate proofs successively, which all return 1 if given the source code to tdt_utility. The implementation of you() could be simply "return 2".

Comment author: 04 June 2012 02:11:56AM 0 points [-]

Thanks for this. I hadn't seen someone pseudocode this out before. This helps illustrate that interesting problems lie in the scope above (callers to tdt_uility() etc) and below (implementation of tdt() etc).

I wonder if there is a rationality exercise in 'write pseudocode for problem descriptions, explore the callers and implementations'.