There are several modes by which that could fail. For example, if the beings have simply mastered a classifier indistinguishable from a typical population member in polynomial time under an adaptive interactive proof protocol (similar to the so-called "Turing Test"), while actually implementing a (source-code-uninspectable) program hostile to that value system.
One person, one vote - a fundamental principle of our democratic government. But what happens in a world where one person can be copied, again and again?
That is the world described by Robin Hanson's "Em economics". Ems, or uploads, are human minds instantiated inside software, and hence can be copied as needed. But what is the fate of democratic government in such a world of copies? Can it be preserved? Should it be preserved? How much of it should be preserved? Those are the questions we'll be analysing at the FHI, but we first wanted to turn to Less Wrong to see the ideas and comments you might have on this. Original thoughts especially welcome!
To start the conversation, here are some of the features of idealised democracy (the list isn't meant to be exhaustive or restrictive, or necessarily true about real world democracies). Which of these could exist in an Em world, and which should?
EDIT: For clarification purposes, I am not claiming that democracies achieve these goals, or that these are all desirable. They are just ideas to start thinking about.