Lumifer comments on [LINK] Scott Aaronson on Google, Breaking Circularity and Eigenmorality - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (46)
What this measures is holding socially acceptable beliefs and behaving in a socially acceptable way. I don't know why he thinks this is what morality is. After all, the entire village collectively and joyfully cooperates in burning the witch.
Scott remarks on this himself:
Scott Says:
By the way, this is also related to the argument in "Well-Kept Gardens Die By Pacifism". When we design a system for moderating a web community, we are choosing between "order" and "chaos", not between "good" and "evil".
We can move the power to moderator, to some inner circle of users, to most active users, even to users with most sockpuppets, but we can't just move it to "good". We can choose which kind of people or which kind of behavior gets the most power, but we can't choose that the power will magically disappear if they try to abuse it; because any rule designed to prevent abuse can also be abused. The values have to come from outside of the voting system; from the humans who use it. So at the end, the only reasonable choice is to design the system to preserve the existing power, whatever it is -- allowing change only when it is initiated by the currently existing power -- because the only alternative is to let forces from outside of the garden optimize for their values, again, whatever they are, not only the "good" ones. And yes, if the web community had a horrible values at the beginning, the proper moderating system will preserve them. That's not bug, that's a side-effect of a feature. (Luckily, on the web, you have the easy option of leaving the community.)
In this sense, we have to realize that the eigen-whatever system proposed in the article, if designed correctly (how to do this specifically is still open to discussion), would capture something like "the applause lights of the majority of the influential people", or something similar. If the "majority of the influential people" are evil, or just plain stupid, the eigen-result can easily contain evil or stupidity. It almost certainly contains religion and other irrationality. At best, this system is a useful tool to see what the "majority of influential people" think is morality (as V_V said), which itself is a very nice result for a mathematical equation, but I wouldn't feel immoral for disagreeing with in at some specific points. Also, it misses the "extrapolated" part of the CEV; for example, if people's moral opinions are based on incorrect or confused beliefs, the result will contain morality based on incorrect beliefs, so it could give you a recommendation to do both X and Y, where X and Y are contradictory.
Well yes, and attempting to group all actual or possible individuals into one tribe is a major mistake, one that I think should be given a name. Well, as it turns out, the name I was already going to give it is at least partially in use: False Universalism.
Ethics ought to include some kind of reasoning for determining when some bit of universalism (some universalization of a maxim, in the Kantian or Timeless sense, or some value cohering, in the CEV sense) has become False Universalism, so that the groups or individuals who diverge from each other to the point of incompatibility can be handled as conflicting, rather than simply having the ethical algorithm return the answer that one or the other is Right and the other is Wrong and the Wrong shall be corrected until they follow the values of the Right.
"Handled as conflicting" seems to either mean "all-out war" or at best "temporary putting off of all-out war until we've used all the atoms on our side of the universe".
If the two sides shared your desire to be symmetrically peaceful with other sides whose only point of similarity with them was the desire to be symmetrically peaceful with other sides whose... then Universalism isn't false. That's its minimal case.
And if it does fail, it seems counterproductive for you to point that out to us, because while we're happily and deludedly trying to apply it, we're not genociding each other all over your lawn.
Sorry, when I said "False Universalism", I meant things like, "one group wants to have kings, and another wants parliamentary democracy". Or "one group wants chocolate, and the other wants vanilla". Common moral algorithms seem to simply assume that the majority wins, so if the majority wants chocolate, everyone gets chocolate. Moral constructionism gets around this by saying: values may not be universal, but we can come to game-theoretically sound agreements (even if they're only Timelessly sound, like Rawls' Theory of Justice) on how to handle the disagreements productively, thus wasting fewer resources on fighting each other when we could be spending them on Fun.
Basically, I think the correct moral algorithm is: use a constructionist algorithm to cluster people into groups who can then use realist universalisms internally.
If we're aggregating cooperation rather than aggregating values, we certainly can create a system that distinguishes between societies that apply an extreme level of noncooperation (i.e. killing) to larger groups of people than other societies, and that uses our own definition of noncooperation rather than what the Nazi values judge as noncooperation.
That's not to say you couldn't still find tricky example societies where the system evaluation isn't doing what we want, I just mean to encourage further improvement to cover moral behaviour towards and from hated minorities, and in actual Nazi Germany.
But his own scheme isn't the aggregation of arbitrary values, it's based on rewarding co operation.
Perhaps in-group problems could be fixed with an eigenSinger algorithm that gives extra points to those who cop operate with people they have not cooperated with before, ie widening the circle.
Collectively, perhaps; but joyfully? Speaking out in defense of the witch risks being labeled a witch yourself.
The burning of a witch is a rightful celebration because the community is rid of a serious danger and the spectacle is a welcome diversion from everyday drudgery :-/
In retrospect, so much of my awareness of actual historical European and American witch-scares is so heavily colored by modern politics and fiction — as, I suspect, most folks' is — that I think we would probably be ill-advised to draw many conclusions from it.
Well, we can make it easier. The age of lynchings and the age of photography intersected. Here, take a look at these two photographs and tell me what the mood of the crowd is.
Warning: persons of sensitive nervous disposition should not click on the links as there will be gruesomeness involved.
Photo 1 Photo 2
I'm broadly sympathetic to your point that Aaronson's algorithm is measuring something like conformity or group consensuses rather than morality, but the specific example you picked to garnish that point is dodgy. From Randall Collins's Violence: A Micro-sociological Theory, pages 425-426:
The last photo Collins refers to ("plate 97") sounds like it's your photo 2, and Collins's interpretation of it seems to me to fit better than the one you imply. I wouldn't go as far as to say that all of the "twenty-nine other visible faces range from somber to apprehensive", as a few of the people in the photo are evidently just trying to get a closer look, and a boy near the front seems more intrigued than anything else. Nonetheless, the crowd as a whole doesn't appear joyful or celebratory to me.
Turning to photo 1, I see perhaps ten spectators' faces clearly enough to make a guess at what they're expressing. Going from left to right, I see (1) thick-eyebrowed man in cap in background who looks as if he's whistling while thinking hard about something; (2) foreground man in white shirt with no tie with neutral-ish expression, but maybe happy; (3) man in middle distance with shorter man in front and a boater-wearer behind, the former gazing off to the photographer's right with a worried/pensive look; (4) smirking man in tie; (5) foreground woman in dress with irregular spots who looks surprised/wary; (6) another woman behind her with open mouth, who might be amused or surprised or scandalized; (7) cluster of three foreground women, where the nearest one's face is too blurry for me to interpret, but the two further back both look apprehensive; (8) foreground man with moustache, pointing as he stares intently; and (9) man at right edge with left eye not visible in photo, looking not especially happily to the photographer's right. As in photo 2, although there are happy-looking people present, they are a minority.
I don't believe it's accurate to use these scenes as examples of rightful celebration or joy. fubarobfusco's warning to exercise caution in how we read these photos may be well-advised.
Yes, I know these photos were analyzed quite substantially, but my point is really simple -- it's that the lynchings (and the witch burnings before them) were culturally normal. The were intense events and, of course, brought out a range of emotions, not just joy, but all I'm trying to say is that the majority of people did not see them as something to be ashamed of. It was OK, it was fine, it was moral.
I'm well aware of such things. My question was aimed at the phrase "the entire village" — which is a claim of unanimity. My point was that members of the village who disapproved would have a strong incentive not to express their disapproval.
Also, American lynchings (of the period you refer to) and witch-hunts are not really the same thing, socially speaking — in part because of the supernatural corruption implied by a claim of witchcraft.
Witch-hunts are an ongoing thing, by the way, in a number of parts of the world. In some cases, economic motives are pretty clear; in others, it seems pretty clearly a matter of superstitious fear that isn't really comparable to the self-evident political or sexual-political motives behind American racial lynchings.
Recall the top-level post.
Sure there will be deviants who disapprove. They are, clearly, very bad people.
They are expressions of the morality of the majority which is good by definition, isn't it?