I'm confused by this setting, but as a response to the idea of controlling stronger things (this comes up in agent-simulates-predictor): You control things to the extent their dependence on your actions controls your actions. So it's possible to control superintelligences or other things that should know all about you, but only if you can predict the effect of your possible actions on their properties, which you probably won't be able to do unless they let you, and I don't understand how such problems look from the point of view of the agent with epistemic advantage (how do you let a piece of paper with "Defect" written on it control your decision?). It seems like what matters here are possible explanations for weaker agent being a certain way, and it's these (potentially more powerful) explanations that play the game with stronger opponents in your stead (so you engage whoever wrote "Defect" of that piece of paper, Transparent Newcomb-style).
(See how pointless it is to just broadcast poorly-explained and poorly-understood ideas?)
See how pointless it is to just broadcast poorly-explained and poorly-understood ideas?
Slightly clumsy but the point came across nonetheless. The last sentence is the interesting one:
It seems like what matters here are possible explanations for weaker agent being a certain way, and it's these (potentially more powerful) explanations that play the game with stronger opponents in your stead (so you engage whoever wrote "Defect" of that piece of paper, Transparent Newcomb-style).
The earlier sentences seem to be making a simple concept more complicated than it needs to be. Control just isn't that deep when you reduce it.
Responding to this: http://lesswrong.com/r/discussion/lw/8ys/a_way_of_specifying_utility_functions_for_udt/
I had a similar idea a few months ago that highlights different aspects of the problem which I find confusing. In my version the UDT agent controls bits of Chaitin's constant instead of the universal prior directly, seeing as one of the programs that the oracle (which you can derive from Chaitin's omega) has to solve the halting problem for is the UDT agent's. But since the oracle for the oracle you get from Chaitin's constant depends on the latter oracle's bits, you seem to be able to ambiently control THE ENTIRE ARITHMETICAL HIERARCHY SAY WHAT!? That's the confusing part; isn't your one true oracle supposed to screen you off from higher oracles? Or is that only insofar as you can computably verify?
Anyway I like this theme of controlling computational contexts as it forms a tight loop between agent and environment, something currently lacking. Keep it up comrades!