I don't know if this is what Will has in mind- but it seems plausible that the super intelligences and gods that would be watching out for us might attempt to maximize the instantiations of our algorithms that are under their domain, so that as great a proportion of our future selves as possible will be saved (this story is vaguely Leibnizian). But I don't know that such superbeings would be capable of overcoming their own sheer unlikelihood (though perhaps some subset of such superbeings have infinite capacity to create copies of us?). You can derive a self-interested ethics from this too- if you think you'll be rewarded or punished by the simulator. The choices of the simulators could be further constrained by simulators above them-- we would need an additional step to show that the equilibrium is benevolent (especially given the existence of evil in our universe).

But I'm not at all convinced Tegmark Level 4 isn't utter nonsense. There is big step from accepting that abstract objects exist to accepting that all possible abstract objects are instantiated. And can we calculate anthropic probabilities from infinities of different magnitudes?

(It does add up to atheism, as a modest claim about our own world, the "real world"

Not if there is evidence of angels and demons in our world, and you can interact with them in at least semi-predictably consequential ways. Which basically everyone believes except the goats, because everyone gets evidence except the goats. Doesn't it suck to have a mind-universe that actively encourages you to fall into self-sustaining delusions? Yes, yes it does.

ETA: Apparently it's 2012 now! My resolution: not to fall into self-sustaining delusion! Happy new year LW!

I agree that there are many reasons that prevent us from explicitly exerting significant control, but I'm at least interested in theurgy. Turning yourself into a better institution, contributing only to the support of not-needlessly-suboptimal institutions, etc. In the absence of knowing what "utility function" is going to ultimately decide what justification is for those who care about what the future thinks, I think building better institutions might be a way to improve the probabilities of statistical-computational miracles. I think this with really low probability but it's not an insane hypothesis even if it is literally magical thinking. (The decision theory and physics backing the intuitions are probably sound, it's just that it doesn't have the feel of well-motivatedness yet. It's more one of those "If I have to choose to spend a few hours either reading about dark matter or reading about where decision theory meets human deciion policies I think it's a potentially more fruitful idea to think about the latter" things.)

I really appreciate that you responded at roughly the right level of abstraction. It seems clear that the debate should be over the extent to which thaumaturgy is possible (including thaumaturgy that helps you build FAIs faster) because that's the only way "theism" or "atheism" should affect our decision policy. (Outside of deciding which object level moral principles to pursue. I like traditional Anglican Christianity when it comes to object level morality even if I mostly ignore it.)

In order to write a computer program that actually computes (rather than models) Maxwell's equations you have to write a program that writes out a physical universe, and if you want a program that describes Maxwell's equations then the interpretation you choose is more a matter of pragmatic decision theory than of algorithmic probability theory, at least in practice. (Bounded agents aren't exactly committing an error of rationality when they don't try to act like Homo Economicus; that would be decision theoretically insane.)

But anyway. Specific things in the universe don't seem to be caused by gods. Indeed, that'd be hella unparsimonious: "God chose to add some ridiculous number of bits into His program just to make it such that there was a 'Messiah gets crucified' attractor?". The local universe as a whole, on the other hand, is this whole other thing: there's the simulation argument.

Your comment got voted up to +10 despite Eliezer's argument being a straightforward error of algorithmic probability; I don't know what to do about that and it stresses me out. Does anyone have ideas? It saddens me to see algorithmic probability so regularly abused on LW, but the few corrective posts on the matter, e.g. by Slepnev, don't seem to have permeated the LW memeplex, probably because they're too technical.

All of existence is strong evidence in favor of theism. The existence of an extremely complex system is obviously evidence of an entity capable and willing to create such a system from scratch. For the kind of priors people deal with everyday- things like "Is Amanda Knox Guilty?" or "Will I win the hand of poker?" the evidence of the strength that we have for God's existence would be more than enough to convince us. But the prior for theism (as it is usually formulated) is so laughably, incomprehensibly low all this evidence isn't even enough for a rational person to seriously consider the theistic hypothesis. Will's claim that a low prior for theism "is an abuse of algorithmic probability theory" is the real issue. Now, that prior can be reduced if the hypothesis involves some process by which the entity could come to exist while conserving complexity (in particular, if that entity evolved and then created this universe). Will however seems to believe in something different than the usual simulation hypothesis- he may endorse something like Divine Simplicity which is complete and utter nonsense. Word games and silliness as far as I can tell- or at least smacking of a to-me-untenable moral realism.

It's complicated. The three versions of theism I can immediately think up are I suppose like "some superintelligent agent is computing us and this is important for our decisions", "all superintelligences converge on the same superinteligent supermoral superpowerful decision algorithm-policy", and "all superintelligences converge on the same superintelligent supermoral decision algorithm-policy and this is important for our decisions". In our current state of knowledge these questions are more logical or indexical-the-way-that-word-used-to-make-sense-before-decision-theory than physical (not to say those are fundamentally different kinds of uncertainty, as I believe Nesov likes to point out). So if I start talking about specific facts of the world then I have to start talking about specific facts about logical attractors akin to how fractal structures are attractors for evolving systems, and I can't point to something nice and concrete like the supposed resurrection of Jesus. This makes the debate really rather difficult--a Bayesian debate much more than a scientific one--and not one where inferential distances can be quickly bridged or where convincing arguments can be made with less than many paragraphs of observations about trends of systems or the nature of modern decision theories.