Curious - what other AI depictions are you considering/comparing to? I'm not 100% sure about what my best would be, I find good bits and pieces here and there in several movies (Ex Machina, 2001: A Space Odyssey, even the very cheesy but surprisingly not entirely unserious M3gan) but maybe not a single organic example I'd place above the rest.

"It's like Mother of Learning, but if it was a cozy romance instead of high fantasy."

This still feels like instrumentality. I guess maybe the addition is that it's a sort of "when all you have is a hammer" situation; as in, even when the optimal strategy for a problem does not involve seeking power (assuming such a problem exists; really I'd say the question is what the optimal power seeking vs using that power trade-off is), the AI would be more liable to err on the side of seeking too much power because that just happens to be such a common successful strategy that it's sort of biased towards it.

If what you mean by 'normalize everything' is to only consider the quantum weights (which are finite as mathematical measures) and not the number of worlds, then that seems more a case of ignoring those problems rather than addressing them.

I mean that the amount of universes that is created will be created anyway, just as a consequence of time passing. So it doesn't matter anyway. If your actions e.g. cause misery in 20% of those worlds, then the fraction is all that matters; the worlds will exist anyway, and the total amount is not something you're affecting or controlling.

This third approach is based on the idea that 'worlds' are macroscopic, emergent phenomena created through decoherence (Wallace's book contains a full mathematical treatment of this). This supports both the claim that the number of worlds is indefinite (since it depends on ultimately arbitrary mappings of macroscopic to microscopic states) and the claim that worlds are created through quantum processes (since they are macroscopically indistinguishable before decoherence occurs). My point in the post was that these two claims in combination can avoid the repugnant conclusion via the approach of focusing on the weights.

I honestly don't think decoherence means the worlds are indefinite. I think it means they are an infinite continuum with the cardinality of the reals. Decoherence is just something you observe when you divide system from environment, in reality the Universe should have only a single, always coherent, giant wavefunction.

I feel like branches being in fact an uncountable continuum is essentially a given, at least unless we were to fundamentally rewrite quantum mechanics to use something other than complex numbers with a cardinality of $2^{{\aleph }_0}$. Talking about branches in terms of countable outcomes only makes sense if we group them by measurement outputs for specific discrete observables; but each of the uncountable infinity of worlds will continuously spawn uncountable infinite worlds and that's just something you gotta deal with. If you want to do ethics over this very confusing multiverse your best bet is probably to normalize everything - "adjust for inflation", so to speak.

I also don't think that even if the worlds were countable (and I have seen arguments to the effect of "actually only integer numbers exist and thus if we looked close enough we'd find that all equations and fields etc are discrete-valued") this would make a lot of difference. You making or not making the experiment does not create more branches, it just determines the outcome of branches that would already exist anyway. Assuming that we can purposefully create branches would require defining "measurement" as an actual discrete specific process, which is a much stronger claim (and I don't think any non-objective interpretation of QM really suggests how to do that, though some gesture towards such a thing existing in theory; and objective QM theories do not admit many worlds). "By looking at specific phenomena, sentient beings create new world-lines" would certainly be A Take; if true, it would beget an ethical nightmare, the Quantum Repugnant Conclusion that we all ought to spend all our time collapsing the wavefunctions that result in the most new worlds being created.

(as a side note, have you read Quarantine, by Greg Egan? I won't explain how precisely to avoid spoiling it, but it deals precisely with these sort of questions)

I don't think this is quite the same thing. Most people actually don't want to have to apply moral thought to every single aspect of their lives, it's extenuating. The ones who are willing to, and try to push this mindset on others, are often singularly focused. Yes, bundling people and ideas in broad clusters is itself a common simplification we gravitate towards as a way to understand the world, but that does not prevent people from still being perfectly willing to accept some activities as fundamentally non-political.

Pretty much. It's not "naive" if it's literally the only option that actually does not harm everyone involved, unless of course we want to call every world leader and self-appointed foreign policy expert a blithering idiot with tunnel vision (I make no such claim a priori; ball's in their court).

It's important to not oversimplify things. It's also important to not overcomplicate them. Domain experts tend to be resistant to the first kind of mental disease, but tragically prone to the second. Sometimes it really is Just That Simple, and everything else is commentary and superfluous detail.

Agree 100% with all of this.

There is one thing that comes to mind IMO and that people who argue that "everything is political" and that neutrality is an evil ploy to actually sneak in your evil ideas really underestimate: the point of impartiality as you describe it is to keep things simpler. Maybe a God with an infinite mind could keep in it all the issues, all the complexities, all the nuances simultaneously, and continuously figure out the optimal path. But we can't. We come up with simple rules like "if you're a doctor, you have a duty to cure anyone, not pick and choose" because they make things more straightforward and decouple domains. Doctors cure people. If you do crimes, there's a system dedicated to punish you. But a doctor's job is different, and the knowledge they need to do it has nothing to do with your rap sheet.

The frenzy to couple everything into a single tangle of complexity is driven by the misunderstanding that complacency is the only reason why your ideology is not the winning one, and that if only everyone was forced to think about it all of the time, they'd end up agreeing with it. But in reality, decoupling is necessary mostly because it allows the world to be cognitively accessible rather than driving us into either perpetual decision paralysis or perpetual paranoia (or worse, both). Destroying that doesn't give anyone victory, we just end up all worse off.

But by believing that they automatically become not Catholic any more, according to the definition of Catholic given by the Catholic Boss who is also the only one with the right to make the rules. If they state that openly they are liable to be excommunicated, though of course most of the times no one will care (even in much darker times the Inquisition probably wouldn't come after every nobody who said something blasphemous once).

I think the crux here is the "relative" poverty aspect. Comparison with others is actually really important, it turns out. Going to Disneyland isn't just a net positive; not going to Disneyland can be a negative if your kids expect you to and all their friends are. A lot of human activities are aimed at winning status games with other humans, and in that sense, in our society of abundance, marketing has vastly offset those gains by making sure it's painfully clear which things make you rich and which aren't worth all that much. So basically the Poverty Restoring force is "other people". No matter the actual material conditions there's always going to be by definition a bottom something percentile in status, and they'll be frustrated by this condition and trying to get out of it to earn some respect by the rest of society.