Vladimir_Nesov comments on A Problem About Bargaining and Logical Uncertainty - Less Wrong

23 Post author: Wei_Dai 21 March 2012 09:03PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (45)

You are viewing a single comment's thread. Show more comments above.

Comment author: Wei_Dai 23 March 2012 06:51:45AM 1 point [-]

Do you mean that I won't have enough computing power also later, after the staple maximizer's proposal is stated, or that there isn't enough computing power just during the thought experiment?

You will have enough computing power later.

What does it mean to self-modify if no action is being performed, that is any decision regarding that action could be computed later without any preceding precommitments?

I mean suppose Omega gives you the option (now, when you don't have enough computing power to compute the millionth digit of pi) of replacing yourself with another AI that has a different decision theory, one that would later give control of the universe to the staples maximizer. Should you take this option? If not, what decision theory would refuse it? (Again, from your current perspective, taking the option gives you 1/2 "logical" probability of 10^20 paperclips instead of 1/2 "logical" probability of 10^10 paperclips. How do you justify refusing this?)

Comment deleted 23 March 2012 07:25:18PM *  [-]
Comment author: Wei_Dai 23 March 2012 07:58:35PM 1 point [-]

I don't know what is the correct decision in this situation anymore, or how to think about such decisions.

Good, I'm in a similar state. :)

The problem has ASP-ish feel to it, you're punished for taking too much information into account, even though from the point of view of having taken that information into account, your resulting decision seems correct.

Yes, I noticed the similarity as well, except in the ASP case it seems clearer what the right thing to do is.

Comment author: Vladimir_Nesov 23 March 2012 08:08:30PM 0 points [-]

(Grandparent was my comment, deleted while I was trying to come up with a clearer statement of my confusion, before I saw the reply. The new version is here.)