A new TED talk video just came out by Daniel Suarez, author of Daemon, explaining how autonomous combat drones with a capability called "lethal autonomy" pose a threat to democracy.  Lethal autonomy is what it sounds like - the ability of a robot to kill a human without requiring a human to make the decision.

He explains that a human decision-maker is not a necessity for combat drones to function.  This has potentially catastrophic consequences, as it would allow a small number of people to concentrate a very large amount of power, ruining the checks and balances of power between governments and their people and the checks and balances of power between different branches of government.  According to Suarez, about 70 countries have begun developing remotely piloted drones (like predator drones), the precursor to killer robots with lethal autonomy.

Daniel Suarez: The kill decision shouldn't belong to a robot

One thing he didn't mention in this video is that there's a difference in obedience levels between human soldiers and combat drones.  Drones are completely obedient but humans can throw a revolt.  Because they can rebel, human soldiers provide some obstacles to limit the power that would-be tyrants could otherwise obtain.  Drones won't provide this type of protection whatsoever.  Obviously, relying on human decision making is not perfect.  Someone like Hitler can manage to convince people to make poor ethical choices - but still, they need to be convinced, and that requirement may play a major role in protecting us.  Consider this - it's unthinkable that today's American soldiers might suddenly decide this evening to follow a tyrannical leader whose goal is to have total power and murder all who oppose.  It is not, however, unthinkable at all that the same tyrant, if empowered by an army of combat drones, could successfully launch such an attack without risking a mutiny.  The amount and variety of power grabs a tyrant with a robot army of sufficient power can get away with is unlimited.

Something else he didn't mention is that because we can optimize technologies more easily than we can optimize humans, it may be possible to produce killer robots in less time than it takes to build armies of human soldiers and with less expense than training and paying those soldiers.  Considering the salaries and benefits paid to soldiers and the 18 year wait time on human development, it is possible that an overwhelmingly large army of killer robots could be built more quickly than human armies and with fewer resources.

Suarez's solution is to push for legislation that makes producing robots with lethal autonomy illegal.  There are, obviously, pros and cons to this method.  Another method (explored in Daemon) is that if the people have 3-D printers, then the people may be able to produce comparable weapons which will then check and balance their government's power.  This method has pros and cons as well. I came up with a third method which is here.  I think it's better than the alternatives but I would like more feedback.

As far as I know, no organization, not even MIRI (I checked), is dedicated to preventing the potential political disasters caused by near-term tool AI (MIRI is interested in the existential risks posed by AGI).  That means it's up to us - the people - to develop our understanding of this subject and spread the word to others.  Of all the forums on the internet, LessWrong is one of the most knowledgeable when it comes to artificial intelligence, so it's a logical place to fire up a discussion on this.  I searched LessWrong for terms like "checks and balances" and "Daemon" and I just don't see evidence that we've done a group discussion on this issue.  I'm starting by proposing and exploring some possible solutions to this problem and some pros and cons of each.

To keep things organized, let's put each potential solution, pro and con into a separate comment.

One person, one vote - a fundamental principle of our democratic government. But what happens in a world where one person can be copied, again and again?

That is the world described by Robin Hanson's "Em economics". Ems, or uploads, are human minds instantiated inside software, and hence can be copied as needed. But what is the fate of democratic government in such a world of copies? Can it be preserved? Should it be preserved? How much of it should be preserved? Those are the questions we'll be analysing at the FHI, but we first wanted to turn to Less Wrong to see the ideas and comments you might have on this. Original thoughts especially welcome!

To start the conversation, here are some of the features of idealised democracy (the list isn't meant to be exhaustive or restrictive, or necessarily true about real world democracies). Which of these could exist in an Em world, and which should?

• Democracy grants legitimacy to the government.
• Democracy is fair and egalitarian - each person has a single vote.
• Democracy aligns the interests of the rulers with that of the ruled.
• Democracy is stable - powerful groups can generally seize power within the structure, rather than overthrowing it.
• Democracy allows the competition of governing ideas.
• Democracy often leads to market economies, which generate large wealth.
• Democracy often lead to welfare states, which increase happiness.
• Democracy doesn't need to use certain coercive methods, such as restrictions on free speech, that other systems require to remain stable.
• Democracy stops a particular group from hanging on to power indefinitely, which can reduce corruption, inefficiency and excessive use of state power for private purposes.

EDIT: For clarification purposes, I am not claiming that democracies achieve these goals, or that these are all desirable. They are just ideas to start thinking about.

Link. Nothing surprising considering previous work on the subject, but a good reminder.

A study by three scientists in the American Political Science Review finds that voters are not competent at accurately evaluating incumbent performance and are easily swayed by rhetoric, unrelated circumstances and recent events.

Gregory Huber, Seth Hill, and Gabriel Lenz constructed a 32-round game where players received payments from a computer "allocator." The goal is to maximize the value of those payments.

Halfway through, at round sixteen, the player had to decide whether to get a new allocator or to stick with the old one.

The allocators pay out over a normal distribution based on a randomly selected mean. Getting a new allocator means that a new mean is selected. This was meant to simulate an election based on performance. 

The group ran three experiments where they changed some of the rules of the game in order to find out how voters could be manipulated or confused over performance. Essentially, how good were voters at accurately analyzing the performance of the "allocator?" 

• The first experiment merely alerted the player at round twelve that they would have the chance to pick a new allocator at round sixteen. This "election in November" reminder made the player weight recent performance in rounds 12-16 over earlier performance in rounds 1-12.

• The second experiment involved a lottery held at round eight or round sixteen. The payout was either -5000, 0, or 5000 tokens. The participant was told that the lottery was totally unrelated to the current allocator, but players still rewarded or punished their current allocator based on their lottery performance.

• The third experiment primed the player with a question right before the election. The question took an adapted form of either Ronald Reagan's "Are you better off than you were four years ago?" or John F. Kennedy's "The question you have to decide on November 8 is, is it good enough? Are you satisfied?" 

The conclusion: 

Participants overweight recent performance when made aware of the choice to retain an incumbent closer to election rather than distant from it (experiment 1), allowed unrelated events that affected their welfare to influence evaluations of incumbents (experiment 2), and were influenced by rhetoric to focus less on cumulative incumbent performance (experiment 3).

If you were ever wondering why Congress has a 95% incumbency rate despite an approval rating in the high teens, this study may be worth a read. 

Related to: Voting is like donating thousands of dollars to charity, Does My Vote Matter?

And voting adds legitimacy to it.

Thank you.

#annoyedbymotivatedcognition