However, you can flip this over trivially and come to a terrible conclusion. If Duplication is true, you merely have to simulate a person until they experience a moment of pure hedonic bliss, in some ethically correct manner that everyone agrees is morally good to experience and enjoy. Then, copy the fragment of the simulation covering the experiencing of that emotion, and duplicate it endlessly.

True just if your summum bonum is exactly an aggregate of moments of happiness experienced.

I take the position that it is not.

I don't think one even has to resort to a position like "only one copy counts".

I think the simpler solution is just to use a bounded utility function. There are several things suggesting we do >this, and I really don't see any reason to not do so, instead of going through contortions to make unbounded >utility work.

But that's essentially already the case. Just consider the bound to be 3^^^^3 utilons, or even an illimited number of them. Those are not infinite, but still allow all the situations and arguments made above.

Paradoxes of infinity weren't the issue in this case.

Then you present me with a brilliant lemma Y, which clearly seems like a likely consequence of my mathematical axioms, and which also seems to imply X - once I see Y, the connection from my axioms to X, via Y, becomes obvious.

Seems a lot like learning a proof of X. It shouldn't surprise us that learning a proof of X increases your confidence in X. The mugger genie has little ground to accuse you of inconsistency for believing X more after learning a proof of it.

Granted the analogy isn't exact; what is learned may fall well short of rigorous proof. You may have only learned a good argument for X. Since you assign only 90% posterior likelihood I presume that's intended in your narrative.

Nevertheless, analogous reasoning seems to apply. The mugger genie has little ground to accuse you of inconsistency for believing X more after learning a good argument for it.

The ratio of the information it adds relative to the total available information is not the point. It's a separate modality. It's subject to a different set of noises and systematic distortions.

(This comment is largely repeating something from my blog)

I would suggest storing, along with the brain, a representative "snapshot" of the working brain, possibly an EEG under standardized conditions.

In the cryonics model, storing your EEG's didn't make much sense. When (if) resuscitation "restarted your motor", your brainwaves would come back on their own. Why keep a reference for them?

But plastination assumes from the start that revival consists of scanning your brain in and emulating it. Reconstructing you would surely be done computationally, so any source of information could be fed into the reconstruction logic.

Ideally the plastinated brain would preserve all the information that is you, and preserve it undistorted. But what if it preserved enough information but garbled it? The information got thru but it was ambiguous. There would be no way to tell the difference between the one right answer that reconstructs your mind correctly and many other answers that construct someone or something else.

Having a reference point in a different modality could help a lot. I won't presume to guess how it would best be used in the future, but from an info-theory stance, there's a real chance that it might provide crucial information to reconstruct your mind correctly.

And having an external reference point could provide something less crucial but very nice: verification that the process worked.

It's the file-drawer problem in comic form.

"Somebody would have noticed" is shorthand for a certain argument. Like most shorthand arguments, it can be used well or badly. Using a shorthand argument badly is what we mean by a "fallacy".

A shorthand argument is used well, in my opinion, just if you could expand it to the longhand form and it would still work. That's not a requirement to always do the full expansion. You don't have to expand it each time, nor have 100% confidence of success, nor expand the whole thing if it's long or boring. But expanding it has to be a real option.

Critical questions that arise in expanding this particular argument:

• What constitutes noticing?

  • Would other people who noticed understand what they saw?
  • Further, would they understand it the same way that we do?
    • How much potential is there for their understanding of the same phenomenon to be quite different from ours?
  • Further, if their understanding is similar to ours, would they express it in terms that we would recognize?
    • This could include actions that we recognize as relating to the phenomenon.

• Would we know that they noticed?

  • Motivations: Would people who noticed have strong motivations for letting us know or for not letting others know?
    • Would they want others to see that they noticed?
    • Would they want others to see the phenomenon they noticed?
    • Would they want to do something about it that someone could easily see?
  • Ability:
    • If they did want others to know, could they easily show it?
    • Conversely, if they didn't, could they easily hide it?
  • Who witnesses it:
    • Would they want us in particular to see it (or not see it), as opposed to a select group? For instance, they might write a report about it that you and I probably wouldn't see.
    • If they revealed it to others but not directly to us, what's the likelihood that the information would make its way to us?

• The suppressed premise in that emthymeme is that "Nobody noticed". Since we didn't ask everyone in the world, how did we determine that?

  • What is the population that would have noticed?
  • What sample size did we take?
  • How representative was our sampling?
  • Assuming we have reasonable answers to the above, what level of confidence can we place on our sampling?

One suggestion, instead of putting "proof" in suspicion-quotes, you could say "argument" instead. A proof is just an air-tight argument.

This was proposed as an alternative to GDP, but it's not clear that it actually measures something similar. Even broadly understanding both as attempts to measure human happiness, it doesn't seem similar.

Since we have no access to time-machines, we cannot give anyone a real choice between travelling back to 1700 and staying in 2010. There are no actual consequences to what they choose. So we are not even measuring people's naive preferences, we are just measuring what they like to say or believe about 1700 vs 2010.

Reality is that which, when you stop believing in it, doesn't go away.

Phillip K. Dick