I don't disagree with your analysis. What I'm pointing out is that I haven't seen any workable proposals to prevent this (or another very disagreeable scenario), except for 1) a singleton AI controlling the effective laws of physics in its light cone, or 2) somehow making sure nobody but a single player ("ruler") has the ability to create computer hardware and/or software. In which case, the universe will probably be tiled with that ruler. The incentives to create copies and the resulting evolutionary pressures are too great.
And in the context of all this, ideas like democracy are completely unworkable unless one of these restrictions is implemented.
I'm all for (1). I've yet to see a plausible scenario not involving a singleton AI that isn't, on some level, horrifying.
One person, one vote - a fundamental principle of our democratic government. But what happens in a world where one person can be copied, again and again?
That is the world described by Robin Hanson's "Em economics". Ems, or uploads, are human minds instantiated inside software, and hence can be copied as needed. But what is the fate of democratic government in such a world of copies? Can it be preserved? Should it be preserved? How much of it should be preserved? Those are the questions we'll be analysing at the FHI, but we first wanted to turn to Less Wrong to see the ideas and comments you might have on this. Original thoughts especially welcome!
To start the conversation, here are some of the features of idealised democracy (the list isn't meant to be exhaustive or restrictive, or necessarily true about real world democracies). Which of these could exist in an Em world, and which should?
EDIT: For clarification purposes, I am not claiming that democracies achieve these goals, or that these are all desirable. They are just ideas to start thinking about.