You're right that you can just take whatever approximation you make at the macroscopic level ('sunny') and convert that into a metric for counting worlds. But the point is that everyone will acknowledge that the counting part is arbitrary from the perspective of fundamental physics - but you can remove the arbitrariness that derives from fine-graining, by focusing on the weight. (That is kind of the whole point of a mathematical measure.)
OK I think I see where you're coming from - but I do think the unimaginable bigness of the universe has more 'irrelevance' implications for a consequentialist view which tries to consider valuable states of the universe than for a virtue approach which considers valuable states of yourself. Also if you think the implication of physics is that everything is irrelevant, that seems like an important implication in it's own right, and different from 'normality' (the normal way most people think about ethics, which assumes that some things actually are relevant).
Thanks for the interesting comments.
You're right, I didn't discuss the possibility of infinite numbers of branches, though as you suggest this leads to essentially the same conclusion as I reach in the case of finite branches, which is that it causes problems for consequentialist ethics (Joe Carlsmith's Infinite Ethics is good on this). If what you mean by 'normalize everything' is to only consider the quantum weights (which are finite as mathematical measures) and not the number of worlds, then that seems more a case of ignoring those problems rather than addressing them.
I hope it was clear that I was suggesting a third approach (the number of worlds is neither finite, nor infinite, but indefinite) which does I think address the ethical problems better, since if there is no definite number of worlds then we have a reason to ignore the number of worlds and focus on the weights.
This third approach is based on the idea that 'worlds' are macroscopic, emergent phenomena created through decoherence (Wallace's book contains a full mathematical treatment of this). This supports both the claim that the number of worlds is indefinite (since it depends on ultimately arbitrary mappings of macroscopic to microscopic states) and the claim that worlds are created through quantum processes (since they are macroscopically indistinguishable before decoherence occurs). My point in the post was that these two claims in combination can avoid the repugnant conclusion via the approach of focusing on the weights.
(And when it comes to the virtue-theoretic implications, I've again tried to follow a weight-based approach, and not make assumptions about whether worlds are created or revealed.)
Thanks for the Egan suggestion, yea I love his work though need to read Quarantine more fully. It seems like the most philosophically relevant bit might be the ending which of course is the source of Egan's Law (it all adds up to normality). I also need to read his short story 'Singleton', which I gather is very relevant too.
Very useful post, thanks. While the 'talking past each other' is frustrating, the 'not necessarily disagreeing' suggests the possibility of establishing surprising areas of consensus. And it might be interesting to explore further what exactly that consensus is. For example:
Yann suggested that there was no existential risk because we will solve it
I'm sure the air of paradox here (because you can't solve a problem that doesn't exist) is intentional, but if we drill down, should we conclude that Yann actually agrees that there is an existential risk (just that the probabilities are lower than other estimates, and less worth worrying about)? Yann sometimes compares the situation to the risk of designing a car without brakes, but if the car is big enough that crashing it would destroy civilization that still kinda sounds like an existential risk.
I'm also not sure the fire department analogy helps here - as you note later in the paper Yann thinks he knows in outline how to solve the problem and 'put out the fires', so it's not an exogenous view. It seems like the difference between the fire chief who thinks their job is easy vs the one who thinks it's hard, though everyone agrees fires spreading would be a big problem.
I'd be interested to know more about the make-up of the audience e.g. whether they were AI researchers or interested general public. Having followed recent mainstream coverage of the existential risk from AI, my sense is that the pro-X-risk arguments have been spelled out more clearly and in more detail (within the constraints of mainstream media) than the anti-X-risk ones (which makes sense for an audience who may not have previously been aware of detailed pro- arguments, and also makes sense as doomscroll clickbait). I've seen lot of mainstream articles where the anti-X-risk case is basically 'this distracts from near-term risks from AI' and that's it.
So if the audience is mainly people who have been following mainstream coverage it might make sense for them to update very slightly towards the anti- case on hearing many more detailed anti- arguments (not saying they are good arguments, just that they would be new to such an audience) even if the pro-X-risk arguments presented by Tegmark and Bengio were strong.
It maybe better first to force person to undergo a course with psychologist and psychiatrist and only after that allow a suicide..
Something along these lines seems essential. It may be better to talk of the right to informed suicide. Arguably being informed is what makes it truly voluntary.
Forcing X to live will be morally better than forcing Y to die in circumstances where X's desire for suicide is ill-informed (let's assume Y's desire to live is not). X's life could in fact be worth living - perhaps because of a potential for future happiness, rather than current experiences - but X might be unable to recognise that fact.
It may be that a significant portion of humanity has been psychologically 'forced' to live by an instinct of self-preservation, since if they stopped to reflect they would not immediately find their lives to be worth living. That would be a very good thing if their lives were in fact worth living in ways that they could not intellectually recognise.
I don't get why you would say that the preferences are fine-grained, it kinda seems obvious to me that they are not fine-grained. You don't care about whether worlds that are macroscopically indistinguishable are distinguishable at the quantum level, because you are yourself macroscopic. That's why branching indifference is not arbitrary. Quantum immortality is a whole other controversial story.