cousin_it comments on Does Solomonoff always win? - Less Wrong

11 Post author: cousin_it 23 February 2011 08:42PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (55)

You are viewing a single comment's thread. Show more comments above.

Comment author: cousin_it 24 February 2011 12:09:23AM *  1 point [-]

From the outside, counterfactual consequences don't appear consistent. If the agent actually chooses action A, the idealized UDT-AIXI thingy will see that choosing action B would have given the agent a billion dollars, and choosing C would have given a trillion. Do you see a way around that?

Comment author: Vladimir_Nesov 24 February 2011 12:46:03AM 0 points [-]

UDT-AIXI could ask which moral arguments the agent would discover if it had more time to think. It won't of course examine the counterfactuals of a fact known to the context in which the resulting mathematical structure is to be interpreted. You can only use a normative consideration from the inside, so whenever you step outside, you must also shift the decision problem to allow thinking about moral considerations.