= 27d567bc2d48d9f40e9ccc20ea370b15)
Houshalter comments on Open thread, Oct. 19 - Oct. 25, 2015 - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (198)
I've been thinking about some of the issues with CEV. It's come up a few times that humanity might not have a coherent, non-contradictory set of values. And the question of how to come up with some set of values that best represents everyone.
It occurs to me that this might be a problem mathematicians have already solved, or at least given a lot of thought. In the form of voting systems. Voting is a very similar problem. You have a bunch of people you want to represent fairly, and you need to select a leader that best represents their interests.
My favorite alternative voting system is the Condorcet Method. Basically it compares each candidate in a 1v1 election, and selects the candidate that would have won every single election.
It is possible for there not to be a Condorcet winner. If the population has circular preferences. Candidate A > Candidate B > C > A... Like a rock paper scissors thing.
To solve this there are a number of methods developed to select the best compromise. My favorite is Minimax. It selects the candidate who's greatest loss is the least bad. I think that's the most desirable way to pick a winner, and it's also super simple.
There are some differences. Instead of a leader, we want the best set of values and policies for the AI to follow. And there might not be a finite set of candidates, but an infinite number of possibilities. And actually voting might be impractical. Instead an AI might have to predict what you would have voted, if you knew all the arguments and had much time to think about it and come to a conclusion. But I think it can still be modeled as a voting problem.
Now this isn't actually something we need to figure out now. If we somehow had an FAI, we could probably just ask it to come up with the most fair way of representing everyone's values. We probably don't need to hardcode these details.
The bigger issue is why would the person or group building the FAI even bother to do this? They could just take their own CEV and ignore everyone elses. And they have every incentive to do this. It might even be significantly simpler than trying to do a full CEV of humanity. So even if we do solve FAI, humanity is probably still screwed.
EDIT: After giving it some more thought, I'm not sure voting systems are actually desirable. The whole point of voting is that people can't be trusted to just specify their utility functions. The perfect voting system would be for each person to give a number to each candidate based on how much utility they'd get from them being elected. But that's extremely susceptible to tactical voting.
However with FAI, it's possible we could come up with some way of keeping people honest, or peering into their brains and getting their true value function. That adds a great deal of complexity though. And it requires trusting the AI to do a complex, arbitrary, and subjective task. Which means you must have already solved FAI.
If I were God of the World, I would model the problem as more of a River Crossing Puzzle. How do you get things moving along when everyone on the boat wants to kill each other? Segregation! Resettling humanity mapped over a giant Venn diagram is trivial once we are all uploaded, but it also runs into ethical problems; just as voting and enacting the will of the majority (or some version thereof) is problematic, so is setting up the world so that the oppressor and the oppressee will never be allowed meet. However, in my experience people are much happier with rules like "you can't go there" and much less happier with rules like "you have to do what that guy wants". This is probably due to our longstanding tradition of private property.
This makes some assumptions as to what the next world will look like, but I think that it is a likely outcome -- it is always much easier to send the kids to their rooms than to hold a family court, and I think a cost/benefit analysis would almost surely show that it is not worth trying to sort out all human problems as one big happy group.
Of course, this assumes that we don't do something crazy like include democracy and unity of the human race as terminal values.
This puts me in mind of Eliezer's "Failed Utopia #4-2".
Not quite.
The local population consists of 80% blue people and 20% orange people. For some reason, the blue people dislike orange people. A blue leader arises who says "We must kill all the orange people and take their stuff!" Well, it's an issue, and how do people properly decide on a policy? By voting, of course. Everyone votes and the policy passes by simple majority. And so the blue people kill all the orange people and take their stuff. The end.
This is exactly the type of problems that mathematicians have tried to solve with different voting schemes. One recent example that has the potential to solve this problem is quadratic vote buying, which takes into account strong preferences of minorities.
I am not sure this is a mathematical problem. Generally speaking, giving a minority the veto power trades off minority safety against government ability to do things. In the limit you have decision making by consensus which has obvious problems.
What do you buy votes with? Money? Then it's an easy way for the blue people to first take orange people's stuff and then, once the orange people run out of resources to buy votes with, to kill them anyway.
That's precisely why it is a mathematical problem... you need to quantify the tradeoffs, and figure out which voting schemes maximize different value schemes and utility functions. Math can't SOLVE this problem because it's a ought problem, not an is problem.
But you can't answer the ought side of things without first knowing the is side.
In terms of quadratic vote buying, money is only one way to do it, another is to have an artificial or digital currency just for vote buying, for which people get a fixed amount for the year.
I don't think your concept of it really makes sense in the context of modern government with a police force, international oversight, etc. All voting schemes break down when you assume a base state of anarchy - but assuming there's already a rule of law in place, you can maximize how effective those laws are (or the politicians who make them) by changing your voting rules.
Ahem.
I would be quite interested to learn who exerts "international oversight" over, say, USA.
Besides, are you really saying a "modern" government can do no wrong??
I'm sorry, I'm not talking about the executive function of the government which merely implements the laws, I'm talking about the legislative function which actually makes the laws. There is no assumption of the base state of anarchy.
This isn't helpful. There's nothing for me to respond to.
The UN (specifically, other very powerful countries that trade with the US).
Would a historical example of what you're talking about be the legality of slavery?
Let me unroll my ahem.
You claimed this is a mathematical problem, but in the next breath said that math can't solve it. Then what was the point of claiming it to be a math problem in the first place? Just because dealing with it involves numbers? That does not make it a math problem.
LOL. Can we please stick a bit closer to the real world?
Actually, the first example that comes to mind is the when the US decided that all Americans who happen to be of Japanese descent and have the misfortune to live on the West Coast need to be rounded up and sent to concentration, err.. internment camps.
Problems can have a mathematical aspect without being completely solvable by math.
I'm not sure that's a fair problem to ascribe to voting. If >50% of that populace wants to kill the orange folks its going to happen, however they select their leaders. It isn't voting's fault that this example is filled with maniacs.