TL;DR by lavalamp: Treating "computers running minds" as discrete objects might cause a paradox in probability calculations that involve self-location. "The probability of being a certain mind" is probably an extensive physical quantity, i.e. rises proportionally to the size of the physical system doing the associated computations.

There are two computers simulating two minds. At some time, one of the minds is being shown a red light, and the other one is shown a green one (call this "Situation 1"). Conditioned on you being one of the minds, what is the probability you should assign to seeing red?

Naively, the answer seems to be 1/2, which comes from assigning being each of the minds an equal probability. If one had three computers and showed two of them a red light and the third one a green one, the probability would be calculated as 2/3, even if the red-seeing computers will be in exactly the same computational state at all times (call this "Situation 2").

However, I think that taking this point of view leads to paradoxes.

An example: Consider an electrical circuit made of (ideal) wires, resistors, capacitors and transistors (sufficient in principle to build a computer); the supply voltage comes from outside of the circuit considered. Under assumptions regarding the physical implementation of this circuit that do not restrict the possible circuit diagrams, it is possible to split the matter composing it into two part that both comprise working circuits reproducing the original circuit's behavior independently of the other part, in an analogous fashion to how the Ebborian's brains are split.* To clarify, what I have in mind is cutting up the wires and resistors orthogonally to their cross-sections - after the splitting, equivalent wires should be on equivalent potentials at the same time, but the currents flowing will be reduced by some factor.

Now imagine the circuit is a computer, simulating the mind that is going to see red in Situation 1 (the mind that will see green still exists). If one splits the circuit as described, one suddenly ends up with two circuits simulating the same mind, i.e. Situation 2 (let's imagine that the computers are split before they are turned on for the first time, so that stream-of-consciousness-considerations will not influence the calculated probability, like e.g. Deda answering 1/2 to Yu'el's question in the linked article). However, it is not clear how far the circuit components need to be apart from each other so that they should be considered "split". I.e., if one fixes a direction in which the circuits are moved apart and then defines P(d) as "the probability one should assign to seeing red, as a function of the distance by which the circuits have been moved apart), P(0) would be 1/2 and P(∞) would be 2/3 in the naive model, but there seems to be no intuitive way how the function should look like in between.

I think that therefore, it is more plausible that a way closer to the correct one to calculate the probability of having one mind's experiences involves somehow weighting this probability by the amount (maybe mass or electron count) of matter that calculates the mind. If one does this, after splitting, the matter comprising each of the parts will add up exactly to the matter of the original circuit, so P(d) would be constant over all distances.

What do you think?

*Namely, the resistors could be full cylinders with the wires protruding along the axes - one could then split them by a plane surface surface that includes the cylinder's axis and would end up with two resistors that have twice the resistance.

The capacitors could look exactly like in this picture and could then be split up along a plane that includes the wires, so that the capacitance is halved.

The transistors could look exactly like in this picture (being homogeneous in the z-direction), and be split up in half across a plane that is parallel to the picture shown).

If one does all of those splittings and splits up the wires so that the parts of each electronic component are connected in the same way as the original circuit was connected, and then operates the resulting circuits with the same supply voltage as one operated the original circuit, the voltages of all wires will always be the same as in the original circuit, and currents will be halved.

Many people see themselves in various groups (member of the population of their home country, or their social network), and feel justified in caring more about the well-being of people in this group than about that of others. They will argue with reciprocity: "Those people pay taxes in our country, they are entitled to more support from 'us' than others!" My question is: Is this inconsistent with some rationality axioms that seem obvious? What often-adopted or reasonable axioms are there that make this inconsistent?

The idea is to compare not the results of actions, but the results of decision algorithms. The question that the agent should ask itself is thus:

"Suppose everyone¹ who runs the same thinking procedure like me uses decision algorithm X. What utility would I get at the 50th percentile (not: what expected utility should I get), after my life is finished?"
Then, he should of course look for the X that maximizes this value.

Now, if you formulate a turing-complete "decision algorithm", this heads into an infinite loop. But suppose that "decision algorithm" is defined as a huge table for lots of different possible situations, and the appropriate outputs.

Let's see what results such a thing should give:

• If the agent has the possibility to play a gamble, and the probabilities involved are not small, and he expects to be allowed to play many gambles like this in the future, he should decide exactly as if he was maximizing expected utility: If he has made many decisions like this, he will get a positive utility difference in the 50th percentile if and only if his expected utility from playing the gamble is positive.
• However, if Pascal's mugger comes along, he will decline: The complete probability of living in a universe where people like this mugger ought to be taken seriously is small. In the probability distribution over expected utility at the end of the agent's lifetime, the possibility of getting tortured will manifest itself only very slightly at the 50th percentile - much less than the possibility of losing 5 Dollars.

The reason why humans will intuitively decline to give money to the mugger might be similar: They imagine not the expected utility with both decisions, but the typical outcome of giving the mugger some money, versus declining to.

¹I say this to make agents of the same type cooperate in prisoner-like dilemmas.

As everyone here knows, it would be a stupid idea to switch from airplanes to cars out of safety/terrorism concerns: Cars are a much more risky means of transportation than airplanes. But what other major risks are there that many people systematically undervalue or are not even consciously aware of?

The same can be asked for chances.