Robin Hanson has done a great job of describing the future world and economy, under the assumption that easily copied "uploads" (whole brain emulations), and the standard laws of economics continue to apply. To oversimplify the conclusion:

• There will be great and rapidly increasing wealth. On the other hand, the uploads will be in Darwinian-like competition with each other and with copies, which will drive their wages down to subsistence levels: whatever is required to run their hardware and keep them working, and nothing more.

The competition will not so much be driven by variation, but by selection: uploads with the required characteristics can be copied again and again, undercutting and literally crowding out any uploads wanting higher wages.

Megadeaths

Some have focused on the possibly troubling aspects voluntary or semi-voluntary death: some uploads would be willing to make copies of themselves for specific tasks, which would then be deleted or killed at the end of the process. This can pose problems, especially if the copy changes its mind about deletion. But much more troubling is the mass death among uploads that always wanted to live.

What the selection process will favour is agents that want to live (if they didn't, they'd die out) and willing to work for an expectation of subsistence level wages. But now add a little risk to the process: not all jobs pay exactly the expected amount, sometimes they pay slightly higher, sometimes they pay slightly lower. That means that half of all jobs will result in a life-loving upload dying (charging extra to pay for insurance will squeeze that upload out of the market). Iterating the process means that the vast majority of the uploads will end up being killed - if not initially, then at some point later. The picture changes somewhat if you consider "super-organisms" of uploads and their copies, but then the issue simply shifts to wage competition between the super-organisms.

The only way this can be considered acceptable is if the killing of a (potentially unique) agent that doesn't want to die, is exactly compensated by the copying of another already existent agent. I don't find myself in the camp arguing that that would be a morally neutral or positive action.

Pain and unhappiness

The preceding would be mitigated to some extent if the uploads were happy. It's quite easy to come up with mental pictures of potential uploads living happy and worthwhile lives. But evolution/selection is the true determiner of the personality traits of uploads. Successful uploads would have precisely the best amount of pain and happiness in their lives to motivate them to work at their maximum possible efficiency.

Can we estimate what this pain/happiness balance would be? It's really tricky; we don't know exactly what work the uploads would be doing ("office work" is a good guess, but that can be extraordinarily broad). Since we are in extreme evolutionary dis-equilibrium ourselves, we don't have a clear picture of the best pain/happiness wiring for doing our current jobs today - or whether other motivational methods could be used.

But if we take the outside view, and note that this is an evolutionary processes operating on agents at the edge of starvation, we can compare this with standard Darwinian evolution. And there the picture is clear: the disequilibrium between happiness and pain in the lives of evolved beings is tremendous, and all in the direction of pain. It's far too easy to cause pain to mammals, far too hard to cause happiness. If upload selection follows broadly similar processes, their lives will be filled with pain far more than they will be filled with happiness.

All of which doesn't strike me as a good outcome, in total.