The story makes two claims about decision theory.
Can you spell out the two claims?
This claim leads to odd questions which cast doubt on it. (Note that the linked post itself links to a better trap in the comments.)
The first objection in the post holds; my decisions are not acausally connected to those of my ancestors in a way that would provide a valid reason to act differently. How I respond to that LW post is a question that never came up in the ancestral environment; only decisions not caused by thinking about decision theory can control whether I exist.
In this specification of transparent Newcomb, one-boxing is correct.
If the argument does work, it would apply to hypothetical entities with certain (highly desirable) powers in some forms of Many-Worlds.
So you're saying that it could also explain the equivalent Fermi-like paradox that asks why beings with Everett-branch jumping powers haven't interfered with us in any way? I agree that, if it explains the Fermi paradox it applies to this scenario too, but I think it is much more likely that Everett-branch jumping is just impossible, as it is according to our current understanding of QM.
I agree that, if it explains the Fermi paradox it applies to this scenario too, but I think it is much more likely that Everett-branch jumping is just impossible, as it is according to our current understanding of QM.
Yes, the argument would only remove a reason for seeing this as a strict logical impossibility (for us).
Can you spell out the two claims?
Sufficiently smart AGI precommits to cooperate with every other super-intelligence it meets that has made a similar precommitment. This acausally ensures that a big set of super-minds will cooperate w
Suppose we could look into the future of our Everett branch and pick out those sub-branches in which humanity and/or human/moral values have survived past the Singularity in some form. What would we see if we then went backwards in time and look at how that happened? Here's an attempt to answer that question, or in other words to enumerate the not completely disastrous Singularity scenarios that seem to have non-negligible probability. Note that the question I'm asking here is distinct from "In what direction should we try to nudge the future?" (which I think logically ought to come second).
Sorry if this is too cryptic or compressed. I'm writing this mostly for my own future reference, but perhaps it could be expanded more if there is interest. And of course I'd welcome any scenarios that may be missing from this list.