Vladimir_Nesov comments on Open Thread June 2010, Part 3 - Less Wrong

6 Post author: Kevin 14 June 2010 06:14AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (606)

You are viewing a single comment's thread. Show more comments above.

Comment author: Vladimir_Nesov 21 June 2010 10:55:00AM *  6 points [-]

Interesting. This suggests thinking about FAI not as using its control to produce terminal value in its own world, but as using its control to buy as much terminal value as it can, in various world-programs. Since it doesn't matter where the value is produced, most of the value doesn't have to be produced in the possible worlds with FAIs in them. Indeed, it sounds unlikely that specifically the FAI worlds will be optimal for FAI-value optimization. FAIs (and the worlds they control) act as instrumental leverage, a way of controlling the global mathematical universe into having more value for our preference.

Thus, more FAIs means stronger control over the mathematical universe, while more UFAIs mean that the mathematical universe is richer, and so the FAIs can get more value out of it with the same control. The metaphors of trade and comparative advantage start applying again, not on the naive level of cohabitation on the same world, but on the level of the global ontology. Mathematics grants you total control over your domain, so that your "atoms" can't be reused for something else by another stronger agent, and so you do benefit from most superintelligent "aliens".

Comment author: Wei_Dai 21 June 2010 11:29:07AM *  5 points [-]

Yes, assuming that trading across possible worlds can be done in the first place. One thing that concerns me is the combinatorial explosion of potential trading partners. How do they manage to "find" each other?

Comment author: Vladimir_Nesov 21 June 2010 11:59:21AM *  4 points [-]

It's the same combinatorial explosion as with the future possible worlds. Even though you can't locate individual valuable future outcomes (through certain instrumental sequences of exact events), you can still make decisions about your actions leading to certain consequences "in bulk", and I expect the trade between possible worlds can be described similarly (after all, it does work on exactly the same decision-making algorithm). Thus, you usually won't know who are you trading with, exactly, but on the net estimate that your actions are in the right direction.

Comment author: JoshuaZ 17 May 2012 07:14:39PM 1 point [-]

Isn't the set of future worlds with high measure a lot smaller?

Comment author: Vladimir_Nesov 17 May 2012 08:40:07PM *  3 points [-]

I currently agree it's a bad analogy and I no longer endorse the position that global acausal trade is probably feasible, although its theoretical possibility seems to be a stable conclusion.