Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

[Link] Dreaming of Political Bayescraft

1 Post author: Zack_M_Davis 06 March 2017 08:41PM

Comments (47)

Comment author: Lumifer 06 March 2017 09:10:05PM 4 points [-]

Why do you equate "genuinely appealing to some feature of human psychology" with benevolence? Killing a rival is genuinely appealing to some feature of human psychology, so is "to crush your enemies, to see them driven before you, and to hear the lamentations of their women".

Comment author: Zack_M_Davis 06 March 2017 09:56:46PM 2 points [-]

I don't expect the joy of crushing one's enemies to be stable under reflection. If you really know deep down that your enemies are just humans like you with a slightly different set of genes who grew up in a different culture, crushing them probably seems less like a good idea, on the grounds that the badness of the subjective experience of suffering being crushed shouldn't depend on whether that experience gets implemented in your own skull or somewhere else (e.g., your enemy's skull). At the same time, however, the underlying features of human psychology that make crushing one's enemies seem like a good idea if you don't know that much philosophy, are probably not things that we want to give up, because they're probably also what make video games and team sports fun.

Comment author: Lumifer 06 March 2017 10:18:28PM *  7 points [-]

I don't expect the joy of crushing one's enemies to be stable under reflection.

I expect the joy of crushing one's enemies to be very stable since it matches the biologically hardwired incentives: you want to perpetuate your genes, not the genes of your neighbour.

crushing them probably seems less like a good idea, on the grounds that the badness of the subjective experience of suffering being crushed shouldn't depend on whether that experience gets implemented in your own skull or somewhere else (e.g., your enemy's skull)

Boggle. Have you, um, looked around at empirical reality? You can theorycraft about perfectly compassionate bodhisattvas who treat the suffering of each sentient as their own, but in the world I live in people crush each other's skulls for power and wealth on a very very regular basis (sometimes with clubs and sometimes with drones) and such practices don't seem to be going away.

The mainstay of government power -- including in the West -- is still the explicit threat of violence. If you piss off the government enough, men with guns will come and force you into compliance. These men are very clear about the difference between your suffering and their suffering.

Comment author: Zack_M_Davis 06 March 2017 10:34:42PM 1 point [-]

the world I live in people crush each other's skulls for power and wealth on a very very regular basis and such practices don't seem to be going away.

Yes, but I'm thinking that if they knew of some clever plan to get the same amount of power and wealth but without the skull-crushing, a lot of people would take it.

The mainstay of government power -- including in the West -- is still the explicit threat of violence.

Unfortunately, that part's never going away: might makes right. But as people and systems made of people continue to exert optimization power on the world, it tends to become increasingly counterfactual violence: if you can predict the outcome of a conflict, you and your adversary can just implement that outcome without bothering to pay the cost of actually fighting.

These men are very clear about the difference between your suffering and their suffering.

Yes, it's probably more profitable for you to model them as extensions of "the government" rather than agents in themselves.

Comment author: Lumifer 07 March 2017 01:54:41AM *  3 points [-]

if they knew of some clever plan to get the same amount of power and wealth but without the skull-crushing, a lot of people would take it

Some would, yes. But there is a slight problem -- such people want more power than what's freely available ("there can be only one"). This means there's competition for power and at this point willingness to crush skulls becomes a rather important advantage.

it tends to become increasingly counterfactual violence

Careful with terminology. That's not counterfactual violence -- that's potential violence. It is nicely encompassed by the traditional phrase "Your money or your life". If I'm holding a gun to your head, of course it's convenient for me if you are going to obey me unconditionally without me having to pull the trigger.

more profitable for you to model them as extensions of "the government"

"Profitable" is probably the wrong word :-) But wasn't your point that if they're enlightened enough, they would refuse to carry out the government's orders?

Comment author: Zack_M_Davis 07 March 2017 02:08:39AM 1 point [-]

But wasn't your point that if they're enlightened enough, they would refuse to carry out the government's orders?

No, I'm thinking that if someone understood history, human psychology, and the game-theoretic Bayes-structure of the universe well enough, they might be able to understand how and why currently-existing governments evolved, and then use that knowledge to engineer new institutions that do a better job of being powerful, stable, and protecting human values?

But this is just me thinking out loud. If this comment isn't useful or interesting, you should downvote it!

Comment author: gjm 07 March 2017 03:10:47AM 2 points [-]

you should downvote it

Downvoting has been disabled for some time.

Comment author: Zack_M_Davis 07 March 2017 03:13:37AM 1 point [-]

Huh, so it is! I've been away for a while!

Comment author: Lumifer 07 March 2017 02:16:44AM 1 point [-]

then use that knowledge to engineer new institutions that do a better job of being powerful, stable, and protecting human values

Yeah, Asimov called that psychohistory :-) Marxists were doing precisely that: they thought they understood how and why the social order evolved and they used that knowledge to engineer new institutions. How did it work out?

However "engineering" is not enough. So you come up with new institutions -- you still need power to implement them. How are you going to acquire this power?

But this is just me thinking out loud.

That's a good thing. Discussion is good, punishing wrong-thinkers is bad.

Comment author: komponisto 07 March 2017 01:02:38AM 0 points [-]

might makes right

Might is perhaps a necessary condition for right, but I would not be inclined to call it a sufficient one.

Comment author: Zack_M_Davis 07 March 2017 01:23:27AM 1 point [-]

... says the guy whose ancestors successfully survived and reproduced.

Comment author: komponisto 07 March 2017 01:25:18AM 2 points [-]

Oh, you meant "might made right".

Comment author: Zack_M_Davis 07 March 2017 01:41:10AM 2 points [-]

Yes, that's a better way of putting it, thanks.

Comment author: username2 07 March 2017 06:18:07PM 1 point [-]

I think you missed the point komponisto was making.

Comment author: fubarobfusco 07 March 2017 06:02:01PM 1 point [-]

See Scott's "The Goddess of Everything Else" for a poetical exposition on the subject.

Comment author: gjm 06 March 2017 10:46:22PM 0 points [-]

people crush each other's skulls for power and wealth on a very very regular basis

Some people do, yeah. But I, for one, have never crushed anyone's skull for power and wealth (mine or anyone else's), and I'm not sure I know anyone who has, and the net quantity of skullcrushing per person seems to be decreasing over time despite the extremely nasty skullcrushing bonanzas of the early-to-mid 20th century.

And while my life has its sorrows and troubles, none of them seems to me to have anything to do with not getting to crush enough skulls. And, again, this seems to me to be true of most of the people I know. So at least within my nice liberal middle-class bubble -- which is, y'know, actually a pretty big bubble, encompassing a substantial fraction of the population -- the "joy of crushing one's enemies" seems not to be much of a thing any more.

Of course, if you allow for metaphorical crushing (sports, board games, getting promoted ahead of the other guy) then there's more of it about. But you have to extend the sense of "crushing" rather a lot for that, and the fact that people seldom actually die from losing at football, chess or office politics is a pretty important fact about those pastimes.

The mainstay of government power [...] is still the explicit threat of violence

Sure. But most of the time the government gets its way without needing to follow through on that threat, and most of the time that's a good thing. (So it seems to me. I am aware that there are political factions that take the opposite view.)

Comment author: Lumifer 07 March 2017 02:00:34AM 4 points [-]

But I, for one, have never crushed anyone's skull for power and wealth

And so, how much power or wealth do you have? Compared to people who were willing to crush skulls.

the net quantity of skullcrushing per person seems to be decreasing over time

That's not obvious to me. I understand that there is considerable debate over the secular trend of violence over timeline measured in centuries.

most of the time the government gets its way without needing to follow through on that threat

Because this threat is real and if you try to call the bluff, the government will make an example out of you.

Comment author: gjm 07 March 2017 03:09:26AM 4 points [-]

how much power or wealth do you have? Compared to people who were willing to crush skulls.

Hard to tell, but I'm much better off than e.g. almost anyone in the armed forces, the great majority of criminals, etc. I'm sure this has much more to do with having marketable skills and above-average ability to delay gratification than with my unwillingness to crush skulls, but I really haven't seen any sign that I'd be much richer or more powerful if I were more inclined towards skullcrushing.

That's not obvious to me.

Nor to me. It was deliberate that I said "seems to be" rather, than, say, "definitely is". I wasn't trying to deny that there's debate.

Comment author: username2 07 March 2017 08:57:36AM 0 points [-]

That's not obvious to me. I understand that there is considerable debate over the secular trend of violence over timeline measured in centuries.

Steven Pinker has put forward the most comprehensive argument I'm aware of in The Better Angels of our Nature in favor of the position that violence is declining over time, no matter how you reasonably measure it. Among the people I know working on this (I've met a few), the uncertainty is over the causes, not the trend itself. But I suppose there could be biases at play still, given that they self-selected to work on these issues.

That all said, I don't think it contradicts the point you are making. There are many reasons that violence is subsiding, as Mr. PInker goes over. But ultimately we are moving over time towards stronger states, and the state's position is based on having a monopoly on violence. When the state has assured its own security, it no longer needs to exercise that power, just threaten it. And after a generation of conditioning, that threat need no longer be brandished -- people just know their roles and fulfill them, with only the deviants being labelled as 'criminals' and having state violence applied against them. So we can expect the amount of violence enacted over time to go down, even as the power asymmetry and implicit threat of violence goes up.

Comment author: Lumifer 07 March 2017 03:40:20PM *  0 points [-]

Steven Pinker

Yes, I think he is the primary proponent of the violence-is-declining argument and his book served as the focus for these discussions.

So we can expect the amount of violence enacted over time to go down, even as the power asymmetry and implicit threat of violence goes up.

That's a valid point, but it applies only to the violence between the state and its subjects. There is also inter-state violence which tends to account for most of the corpses plus raiding/predation by the state outside of its own borders (see the recent American adventures in the Middle East after the Iraq war).

Plus there is also the commonly-ignored distinction between showing that a historical trend existed and making a forecast: saying that this trend will continue in the future. Given that we live in interesting times, I would be wary of assuming that historical trends will just extrapolate.

Comment author: username2 07 March 2017 06:16:08PM 0 points [-]

I suggest reading the book or watching his master course lecture. He deals with exactly these objections in more detail than I'm willing to regurgitate.

Comment author: gjm 07 March 2017 12:59:40PM 0 points [-]

Steven Pinker [...]

I have seen it suggested that Pinker's alleged observation of declining violence is something of a statistical artefact -- violent death has a heavy-tailed distribution, so the usual methods of estimating trends may fail badly.

we can expect the amount of violence enacted over time to go down, even as the power asymmetry and implicit threat of violence goes up

That may be true, but the context here is a discussion of whether we should see joy in crushing one's enemies as an essential and stable part of human nature. I think there's an important difference between joy in crushing one's enemies and the practical expedience of threatening to crush one's enemies, from that perspective.

Comment author: username2 07 March 2017 06:13:46PM 2 points [-]

That may be true, but the context here is a discussion of whether we should see joy in crushing one's enemies as an essential and stable part of human nature.

Taking joy in vengeance / revenge is pretty darn universal. It seems really strange to have to justify this, given the preponderance of evidence in the form of literature, plays, and myth from all cultures.

Comment author: gjm 08 March 2017 03:40:52PM 2 points [-]

Some degree of joy-in-vengeance appears to be built into human biology, yes. Zack suggests that it isn't "stable under reflection": that is, if we were able to modify our values, then even though at present we have some tendency to rejoice in the suffering of our enemies we would (if smart enough) choose to stop doing so.

I think there is some evidence to support this, and in particular to support the idea that we have, on a timescale of centuries, modified our values to involve less rejoicing in the suffering of enemies.

This is one of those discussions where it seems like each comment is addressing a question just slightly different from the previous comment's, and after a while there's severe divergence, so let's take it from the top for context's sake.

  • Zack originally claimed that " the forces of memetic evolution are somewhat benevolent" on the grounds that ideas that spread "have to be genuinely appealing to some feature of human psychology".
  • Lumifer objected that appealing to human psychology and benevelonce are very different things, offering the example of crushing one's enemies.
    • I think this is a reasonable objection.
  • Zack countered that the joy of crushing one's enemies is probably not stable under reflection.
    • I'm not sure this is relevant to the foregoing discussion, but I think it's right. As evidence I remark that most people thinking about values seem to conclude that taking joy in crushing one's enemies is mostly a bad thing. Most systems of ethics either condemn it explicitly or embrace principles that imply condemning it.
  • Lumifer said: no, the joy of crushing one's enemies should be very stable because it or something very like it is biologically hardwired.
    • This looks to me like a very different notion of stability from Zack's: something strongly enough hardwired will be Lumifer-stable even if it's so clearly opposed to our other explicitly-held values as to be definitely not Zack-stable.
    • Whether that's a problem depends on why Zack considered the notion of stability-under-reflection relevant in the first place, which I am not certain of.
    • It also looks empirically wrong to me. For sure, joy-in-crushing-enemies hasn't completely disappeared from human nature (no doubt for exactly the reasons Lumifer gave) -- but it seems to me to have become much less pronounced and much less important in how societies work, over (say) the last couple of thousand years. At least in the societies LW people are almost all part of.
  • Lumifer claimed, more specifically, that literal enemy-crushing -- smashing skulls, and all that -- is still commonplace. (Again, as an objection to Zack's claim that joy-in-crushing-enemies isn't stable under reflection.)
    • This is again clearly concerned with Lumifer-stability (does it in fact persist given human nature?) rather than Zack-stability (would we keep it if we had the choice?).
    • And it also looks empirically doubtful at best. Most of "Western civilization", which of course is far from being the whole world but is a big chunk of it, has treated joy in literal skullcrushing as socially unacceptable in almost all circumstances for quite some time, and this doesn't seem to have done us any particular harm. The existence of skullcrushing in (overt and less-overt) wars far away, and the theoretical threat of skullcrushing for those too obdurately unwilling to obey the law, are irrelevant, since they don't offer most of us any opportunities for actual joyful skullcrushing.

So: sure, we're hardwired for some degree of vengefulness; sure, some amount of (actual or threatened) skullcrushing is necessary for practical reasons. Yet, despite those things, we in the allegedly-civilized West mostly manage to get by with no literal skullcrushing at all and not all that much metaphorical skullcrushing, and our societies are pretty successful by any reasonable measure, more so I think than societies that still embrace skullcrushing as a source of joy.

That is, joy in crushing our enemies, in "our" societies,

  • is generally condemned by people and institutions that have thought hard about values
  • has been quite successfully suppressed
  • without obvious adverse consequences
  • despite biological hardwiring and occasional pragmatic necessity

all of which seems to me to support Zack's contention that it's not stable under reflection.

Comment author: Lumifer 08 March 2017 04:07:27PM *  1 point [-]

That is, joy in crushing our enemies, in "our" societies (*) is generally condemned by people and institutions that have thought hard about values; (*) has been quite successfully suppressed; (*) without obvious adverse consequences; (*) despite biological hardwiring and occasional pragmatic necessity.

Let's try a couple of examples: (a) WW2; (b) Islamic terrorism.

Plus, there is the usual problem of what to do when Joyous Skullcrushers come swinging their clubs.

Comment author: Dagon 07 March 2017 12:29:55AM 1 point [-]

Wait. The fact that the government usually gets it's way with a deeply hidden implicit threat of violence rather than with overt violence matters how? The underlying primate behaviors of jealousy and social power is still there, and it's hard to claim those aren't fundamental features of human psychology.

Comment author: gjm 07 March 2017 03:30:25AM 0 points [-]

The foregoing discussion was about how much joy people take in crushing their enemies, driving them before them, and hearing the lamentation of their women.

Government power may well be founded on the theoretical possibility (and relatively rare reality) of coercive violence (I would suggest, naive special snowflake that I am, that actually the consent of the governed is just as important as the threat of coercive violence, but never mind that) but it doesn't generally involve a great deal of joyful skullcrushing, and joy-in-skullcrushing on the part of the enforcers is generally regarded as a bug rather than a feature and isn't, so far as I can see, actually required in practice.

Comment author: DryHeap 09 March 2017 05:41:01PM 0 points [-]

But I, for one, have never crushed anyone's skull for power and wealth (mine or anyone else's), and I'm not sure I know anyone who has, and the net quantity of skullcrushing per person seems to be decreasing over time despite the extremely nasty skullcrushing bonanzas of the early-to-mid 20th century.

I'm not sure if the user you are replying to meant it like this, but it would make sense that your ancestors did plenty of the "skull-crushing" for you (in a sense), and that is why you are here today. The colonialism and use of violence in the past is why you enjoy the life you do today.

Comment author: gjm 10 March 2017 01:48:05AM 1 point [-]

The colonialism and use of violence in the past is why you enjoy the life you do today.

Maybe so. Also, the single-celled-ness of my distant ancestors was an essential step to the form of life I now have; their primitivity is (part of) why I enjoy the life I do today. This observation doesn't in the least lead me to wish I were a single-celled life form, nor to see single-celled life forms as somehow my superiors, nor to anticipate a future reversion to single cells in my descendants.

If your suggestion is that since some of my ancestors were skullcrushers and I wouldn't be here without them I should look back with admiration to my skullcrushing ancestors, or that I should expect my successors to return to skullcrushing, then I would be interested to know why the argument doesn't apply equally to unicellularity.

Comment author: DryHeap 17 March 2017 03:03:52PM 0 points [-]

I would be interested to know why the argument doesn't apply equally to unicellularity.

I apologize for a misunderstanding. My argument was not that one should "expect [their] successors to return to skullcrushing." It is that, as skull-crushing was an essential part of our evolution, it makes sense that we still have that latent impulse. Think sex.

I ... have never crushed anyone's skull for power and wealth ... and I'm not sure I know anyone who has

The people who actively "skull-crush" for power and wealth are largely tribes of people in the third world. This is occurring in many parts of Africa and some parts of the Middle East. This is where they do not enjoy the same quality of a first-world life as you or I do. Sure, you may not do any skull-crushing. You live a life which is pleasant enough, and the propagation of your lineage is guaranteed enough (and if you do not wish to bear fruit, so to speak, this is another indicator of the pleasantries of the first world). This is missing the point.

So at least within my nice liberal middle-class bubble -- which is, y'know, actually a pretty big bubble, encompassing a substantial fraction of the population -- the "joy of crushing one's enemies" seems not to be much of a thing any more.

First, it's not a large bubble given total human population. It is in fact quite a small bubble standing on the shoulders of violent giants. Maybe we should leave this bubble from time to time.

Second, the joy of crushing you enemies is still innate, as you state yourself ("if you allow for metaphorical crushing ... then there's more of it about"). Just as sexual reproduction leads to competitively viable organisms, leading to our innate desire for sex (even without the explicit desire to reproduce - another wonder of our first world) , violent competition led us to the point we are now. Can you say that you do not enjoy your doses of "metaphorical crushings"? Would you say that, in a doomsday-scenario, the use of violence would be a competitive advantage? If so, then surely some are utilizing it. They would have the advantage. They would reproduce (whether consensual or not - yet again a modern world invention). And we return to our forebears, and the latent spirit of violence lying beneath the surface of our civilized lives...

I digress. I'm not implying that all acts of our ancestors are needed today. I am stating that, no matter one's personal experience with violence, their existence has benefited from previous acts of violence. To state that these impulses have left us is false.

Comment author: gjm 17 March 2017 03:32:54PM 0 points [-]

Can you say that you do not enjoy your doses of "metaphorical crushings"?

Well, speaking purely for myself, my skullcrushing activities never get more non-metaphorical than, say, a game of chess. I've no particular objection to fictional violence, but tend to find it among the least interesting things in the fiction containing it. I will concede that I enjoy winning at chess, but I beg leave to doubt how much evidence that is for the stability of joy-in-actual-skull-crushing.

I'm afraid it seems to me that your comment is long on rhetoric and short on actual reasoning, which makes it difficult to respond to in a way I find adequate. So I'll just repeat in slightly different words what I already said: yes, plenty of skullcrushing in our ancestral past, but I am unconvinced that this fact tells us much about what our future either will or should be like.

Comment author: Dagon 06 March 2017 09:44:04PM 2 points [-]

Being smart is more important than being good (for humans).

Scary wrong. Smart and good are multiplicative factors. The most important one is whichever you can improve by a higher percentage.

Comment author: Zack_M_Davis 06 March 2017 10:07:25PM 1 point [-]

Sure, I consider the "(for humans)" parenthetical to be doing a lot of work. If we interpret "good" as meaning what (some agent-like system) would want if it knew more, thought faster, &c., it's going to be true that some humans are quantitatively less good with respect to the "good" (extrapolated volition) of everyone else. But I expect most people to wildly overestimate the quantitative extent to which this matters; selfishness is a much more powerful force in the world than outright evil. So, you should read me as claiming that smart is improvable by a much higher percentage.

Comment author: Dagon 06 March 2017 10:49:12PM 0 points [-]

I consider the "(for humans)" parenthetical to be doing a lot of work.

that is a lot of work, and probably should be made explicit that you mean "for typical smartness and goodness of my current peer group". Unless you really mean to deny that "goodness" can be zero or negative, and then becomes far more important than smartness.

Comment author: Zack_M_Davis 07 March 2017 01:16:20AM 2 points [-]

should be made explicit that you mean "for typical smartness and goodness of my current peer group".

It turns out that my current peer group does not have a magical monopoly on goodness! It even turns out that being two or three standard deviations from the mean in intelligence does not give us a magical monopoly on goodness! I didn't notice this until very recently!

I think it's a really valuable (if expensive and painful) exercise to spend a lot of time reading the literature of some ideology that you despise, really trying to learn from their models and see the Bayes-structure that they're pointing at, even if you don't like them and don't want to join their group. When the model clicks (this may take a few years), you might learn something! (You still don't have to join the hated outgroup when this happens—you are in fact free to continue to hate them—but the experience may change you enough that you don't fit in with your ingroup anymore.)

Comment author: Lumifer 07 March 2017 02:04:11AM 1 point [-]

it's a really valuable (if expensive and painful) exercise ... you might learn something

Those two thoughts don't seem to match well. If you think it's valuable enough to be worth the expense and the pain, presumably you have a better description of the potential payoff other than "learn something"?

Comment author: Zack_M_Davis 07 March 2017 02:20:33AM 2 points [-]

The payoff is the shock of, "Wait! A lot of the customs and ideas that my ingroup thinks are obviously inherently good, aren't actually what I want now that I understand more about the world, and I predict that my ingroup friends would substantially agree if they knew what I knew, but I can't just tell them, because from their perspective it probably just looks like I suddenly went crazy!"

I know, that's still vague. The reason I'm being vague is because the details are going to depend on your ingroup, and which hated outgroup's body of knowledge you chose to study. Sorry about this.

Comment author: Lumifer 07 March 2017 02:24:46AM 1 point [-]

A lot of the customs and ideas that my ingroup thinks are obviously inherently good, aren't actually what I want now that I understand more

So if my attitude is already this, can I skip the pain and the expense? :-)

Comment author: Sarunas 07 March 2017 02:10:10PM *  1 point [-]

Any time you find yourself being tempted to be loyal to an idea, it turns out that what you should actually be loyal to is whatever underlying feature of human psychology makes the idea look like a good idea; that way, you'll find it easier to fucking update when it turns out that the implementation of your favorite idea isn't as fun as you expected!

I agree that this is a step in the right direction, but I want to elaborate why I think this is hard.

It is my impression that many utopians stay loyal to their chosen tactics that are supposed to closen the utopia even after the efficiency of the those tactics come into question. My hypothesis for why such thing can happen is that, typically, the tactics are relatively concrete whereas goals they are supposed to achieve are usually quite vague (e.g. "greater good"). Thus when goals and tactics conflict the person who tries to reconcile them will find it easier to modify the goals than the tactics, perhaps without even noticing that the new goals may be slightly different from the old goals, since due to vagueness the old goals and the new goals overlap so much. Over time, goals may drift into becoming quite different from the starting point. At the same time, since tactics are more concrete it is easier to notice changes in them.

I suspect that in your case we might observe something similar, since it is often quite hard to pinpoint exactly what underlying features of human psychology make a certain idea compelling.

really smart people who know lots of science and lots of probability and game theory might be able to do better for themselves

I agree that science, probability and game theory put constraints on how hard problems of politics could be solved. Nevertheless, I suspect that those constraints, coupled with vagueness of our desires, may turn out to be lax enough to allow many different answers for most problems. In this case this idea would help to weed out a lot of bad ideas, but it may not be enough to choose among those the rest. In another case, those constraints may turn out to be too restrictive to satisfy a lot of desires people find compelling. Then we would get some kind of impossibility theorem and the question which requirements to relax and what imperfections to tolerate.