Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Recognizing memetic infections and forging resistance memes

4 Post author: Johnicholas 26 April 2012 02:40PM

What does an memetic infection look like? Well, you would encounter something (probably on the internet) that seems very compelling. You think intensely about it for a while, and it spurs you to do something - probably to post something related on the internet. After a while, the meme may not seem that compelling to you anymore, and you wonder why you invested that time and energy. The meme has reproduced itself. For example, Bruce Sterling's response to the 'New Aesthetic' is a paradigmatic example of memetic infection: he encountered it, he found it compelling, he wrote about it, I read about it and now I know about it. (Note that the word 'infection' has a stigma to it, but I don't mean it to be necessarily a bad thing. I will use 'disease' to mean 'infection with bad consequences'.)

Now, let me jump to an apparently unrelated concept - Viral Eukaryogenesis. If I understand correctly, Viral Eukaryogenesis is the theory that eukaryotes (including you and me) are inheritors of a bargain between two kinds of life - metabolic life and viral life, something like the way lichens are a bargain between fungi and algae. The nucleus that characterizes eukaryotes is supposed to be descended from a virus protein shell, and the membrane-fusion proteins that we use for gamete fusion (crucial for sex) are supposed to be descended from viral infection proteins. I am not a biologist, but my understanding of the state of biology is that it is an interesting hypothesis, as yet neither proven nor disproven. However, I'm going to talk as if it were true, because I'm actually trying to make an analogy with memes.

What is the advantage of the bargain between metabolic life and viral life? Perhaps sex. What is the advantage of sex? Perhaps defense against parasites and/or disease. What would a memetic parasite or disease look like? Well, it would be a memetic infection that, on the whole, leaves the individuals it touches worse off. For example, if someone spent so much time propagating the meme that their previous goals were harmed, then they would be suffering from a memetic disease. A cult might be a memetic disease. I have heard that abused children sometimes grow up to be abusers - that would be a memetic disease. Any sort of self-catalyzing pattern of behavior (particularly communication behavior) that on the whole harms the individuals exhibiting it is a memetic disease. A confusion or misunderstanding on a particular point that leads teachers to teach that point confusingly would be a memetic disease; for example, the story about Bernoulli's principle and airplane wings.

Note that just because evolution, genetic and memetic, has been going on for a long time, it doesn't mean that individual diseases are very smart; they can be very young, newly accidentally created and not very evolved. They are not necessarily very infectious. A self-catalyzing pattern within a single human could be a disease - a stimulus, perhaps textual, in a particular person's environment that leads that person to a pattern of thought that results in re-creating or preserving that stimulus is a meme no less for being transmitted from a past self to a future self. A robust pattern of relentless self-criticism reinforced with post-it-notes might well be a memetic disease, even if it doesn't seem likely to transmit itself from one host to another. A slowdown, a decrease in productivity after doing the same thing over and over again, might be caused by a gradual accumulation of parasitic self-catalyzing patterns - memes.

Many memes are self-immunizing - having seen it once leads to recognizing it and not re-transmitting it. It may be that a policy of free speech and rapidly mixing pattern of conversation gives better results than trying to quarantine memes. Still, some memetic diseases keep catching us despite having been caught before. How can we create and spread resistance to memetic diseases? Some memes 'work' (that is, propagate themselves) only if they're implicit - knowing an explicit analysis of how the meme functions in an unwitting host is sufficient to defeat it. This knowledge, if it's transmissible, is a resistance meme.

When you find yourself failing (at any scope - even small failures matter), deliberately write some text, an explicit analysis and explanation of the failure. The text is an attempt at a resistance meme to the cause of the failure. Archive these pieces of text and take good care of them. When you have an opportunity, for example in conversation with like-minded folk, bring these analyses and explanations, and try to shuffle, collate and merge them together into more potent forms - standard checklists and processes and methodologies with links to the mistakes that they were forged from. We fight memetic diseases by forging and spreading memes.

How do we know we're not spreading memetic diseases ourselves? We need to keep archives, something like breeding records - the initial text describing the failure is the start of an archive, a pedigree of a resistance meme. Without the evidence of failures averted, a resistance meme can become a disease in itself. I believe DHH's "Testing like the TSA" is relevant here - test-first development was intended as a resistance meme to certain pernicious, arguably self-catalyzing failures. As it became decoupled from actual failures, it turned into something like a religion; more parasitic than helpful. 

Another strategy might be, when you find yourself failing, simply to try something different, to change policy. Mutation might create a resistance meme, or a continuously changing environment might make you a moving (more difficult) target for parasitic memes. Stop thinking that the norm is stability - the norm is a Red Queen race.

(This post brought to you by Schneier's "Liars and Outliers".)


Possibly irrelevant rant:

Just as us eukaryotes are inheritors of a bargain between metabolic life and viral life, us humans are inheritors of a bargain between eukaryotic life and memetic life. We should not identify with our eukaryotic heritage over and above our memetic heritage. Humancentrism is analogous to racism or sexism. How do you want your descendants to act toward one another after humanity speciates? The boundary of the magic circle of compassion cannot be fixed at co-fertility of the eukaryotic halves of ourselves. I don't know where it ought to be, but fear-fueled bigotry is well-known as a human failure mode. We've formed alliances with utterly alien forms of life in the past, and those have been some of our best successes.

Comments (36)

Comment author: JoshuaZ 26 April 2012 08:56:20PM 8 points [-]

How do you distinguish a meme that is infectious because it is hijacking imperfect hardware and a meme that is actually spreading because it has good content? There are some very simple concepts that spread easily that could be regarded as memes. For example, the idea of Fermi estimates is essentially a meme. But it is one that is useful in many contexts.

Comment author: JGWeissman 26 April 2012 09:04:58PM 7 points [-]

It seems that Johnicolas would call the Fermi estimate meme "infectious", but not a "disease".

(Note that the word 'infection' has a stigma to it, but I don't mean it to be necessarily a bad thing. I will use 'disease' to mean 'infection with bad consequences'.)

It may be better to more explicitly counter the connotations of common usage by calling it a "benign infection".

Comment author: [deleted] 27 April 2012 11:38:30AM 0 points [-]

This reminds me of someone complaining somewhere that the word viral is now used without negative connotation (as in viral advertising by its advocates), and that that's not right.

Comment author: timtyler 27 April 2012 10:07:19PM *  4 points [-]

Natural organic viruses can be good for you. E.g. see the Seneca Valley Virus.

"Viral" isn't wrong - though there is a case that it is potentially misleading. I tend to use "cultural symbiont", "cultural parasite" and "cultural mutualist".

Comment author: Johnicholas 26 April 2012 09:29:52PM -1 points [-]

Thank you. Exactly.

Comment author: timtyler 27 April 2012 10:05:09PM *  3 points [-]

How do you distinguish a meme that is infectious because it is hijacking imperfect hardware and a meme that is actually spreading because it has good content?

A biologist would look at the effect of the cultural symbiote on host fitness - and classify it as a parasite, mutualist or commensalist, accordingly.

Comment author: Vaniver 26 April 2012 05:47:46PM 15 points [-]

Humancentrism is speciesist.

...so? This seems like a statement that weakens your argument, as it attempts to smuggle in implications in a way readers can easily disagree with.

Comment author: Johnicholas 26 April 2012 09:27:57PM -1 points [-]

Thanks, I'll replace the sentence with "Humancentrism is analogous to racism or sexism". It's no defense, but I'm not intentionally trying to use dark arts of persuasion.

Comment author: Jack 26 April 2012 10:32:36PM 15 points [-]

The change makes it worse.

Comment author: [deleted] 27 April 2012 12:30:19PM 5 points [-]

Well... With the present version I get the point, but with the original I would have thought ‘Isn't that a tautology?’. I guess this mean that speciesism doesn't have the negative connotations of sexism or racism in my mental lexicon.

Comment author: faul_sname 27 April 2012 06:10:38AM 9 points [-]

In what way is it analogous, and why should we care?

Comment author: Johnicholas 27 April 2012 11:38:14AM 3 points [-]

Analogous in that people once discriminated against other races, other sexes, but over time with better ethical arguments, we decided it was better to treat other races, other sexes as worthy members of the "circle of compassion". I predict that if and when we interact with another species with fairly similar might (for example if and when humans speciate) then humancentrism will be considered as terrible as racism or sexism is now.

Moral realism (if I understand it correctly) is the position that moral truths like 'eating babies is wrong' are out in the world something like the law of gravitation. Yudkowsky has argued convincingly in the Baby-Eater sequence against moral realism (and I agree with him). However, he implied a false fork that, if moral realism is false, then humancentrism is the answer. Yes, our sense of morality is based on our history. No, our history is not the same as our species.

DNA is one residue of our history, but libraries are also a similar residue. There are two instances in our history of allying with a very alien form of life: Viral eukaryogenesis, and the alliance with memes.

Does this help at all? I feel like I'm saying the same thing over again just with more words.

Comment author: [deleted] 27 April 2012 12:32:14PM *  2 points [-]

However, he implied a false fork that, if moral realism is false, then humancentrism is the answer.

I don't think he would put it that way. He defines good as “that which leads to sentient beings living, to people being happy, to individuals having the freedom to control their own lives, to minds exploring new territory instead of falling into infinite loops, to the universe having a richness and complexity to it that goes beyond pebble heaps, etc.”, not as ‘what humans value’, and considers it a “moral miracle” that humans value what leads to sentient beings living etc. etc. (Of course, the reason why we're talking about what leads to sentient beings living etc. etc. in the first place is that that's what we value, so IMO being surprised that we value that would be --as Feynman put it (though he was talking about something else)-- like being surprised that I can see the car with the number plate AC 443 MW.)

Comment author: Hul-Gil 30 April 2012 04:52:02AM 1 point [-]

I feel like you're trying to say we should care about "memetic life" as well as... other life. But the parallel you draw seems flawed: an individual of any race and sex is still recognizably conscious, and an individual. Do we care about non-sentient life, memetic or otherwise? Should we care?

Comment author: timtyler 26 October 2013 12:06:46PM *  0 points [-]

I feel like you're trying to say we should care about "memetic life" as well as... other life.

I don't know about 'should' - but many humans do act as though they care about their favoured memes.

Catholicism, Islam, patriotism - there are many memes that are literally 'to die for'.

Comment author: Johnicholas 30 April 2012 12:15:17PM *  -1 points [-]

You're right, it's infeasible to care about individual memes (or for that matter, the vast majority of individual animals) the way we care about other humans. I don't have an answer to your question, I'm trying to break a logjam of humancentric ethical thinking.

Forgive me for passing on my confusion here, but I'm not certain that consciousness/sentience, is anything more than 'recognizably human'. You and I have a common brain architecture and one of our faculties is picking that out from trees and rocks. Perhaps there are plenty of evolved, competent alien minds that would pose a challenge to our ape-like-me recognition systems simply because they're so alien.

But if and when humans upload, then they will become effectively memes. We need to solve the question of how to care about non-sentient life, because a datafile could be you or me or a descendent.

Comment author: JoachimSchipper 27 April 2012 12:32:06PM -1 points [-]

Does this have anything to do with the main point of your article? I can find no links, except an unproven theory used only as an analogy.

Comment author: Johnicholas 27 April 2012 02:03:18PM 0 points [-]

No. This is regarding the 'possibly irrelevant rant' which I marked explicitly as a 'possibly irrelevant rant'. The concepts in the rant seemed nearby and inspirational to the main article in my mind when I wrote it, which is why I included it, but I cannot articulate a direct connection.

Comment author: Eugine_Nier 04 May 2014 03:54:38AM 6 points [-]

Humancentrism is analogous to racism or sexism

In the sense that all the above are vaguely defined accusations, that are frequently used to suppress certain statements (many of which are true) and as a result lead to bad policy because it's based on false beliefs? In that case I agree.

Comment author: timtyler 28 April 2012 12:35:11PM *  3 points [-]

Note that the word 'infection' has a stigma to it, but I don't mean it to be necessarily a bad thing. I will use 'disease' to mean 'infection with bad consequences'.

There's also the word "contagion" to consider.

The dictionary entry for that includes:

the spreading of an emotional or mental state among a number of people: the contagion of mirth.

It explicitly includes culture and has weaker implications of being deleterious. I think what we really need is the terminology of generalised epidemiology, though.

Comment author: beoShaffer 26 April 2012 04:50:59PM 3 points [-]

Interesting, it doesn't have as much clear evidence as I would like, especially for the section on creating resistance memes, but still reaches the just try it level. At least for people with time that is, I'm already try to integrate several new positive habits into my life and have several more that I'd like to try, so I don't expect to take up your advice in the near future.

Comment author: JoachimSchipper 27 April 2012 12:54:45PM 3 points [-]

Some constructive criticism (article is at -4 as I write this): although there is place on LW for an article about memes, your article would be clearer without the viral eukaryogenesis metaphor and the irrelevant rant. Your goal-directed ("resistance") memes are an interesting concept; between checklists and lost purposes, there seems to be an interesting article there.

Comment author: timtyler 27 April 2012 10:12:41PM 2 points [-]

Your goal-directed ("resistance") memes are an interesting concept; between checklists and lost purposes, there seems to be an interesting article there.

Those are sometimes called vaccimes in memetics.

Comment author: fubarobfusco 26 April 2012 04:02:43PM *  4 points [-]

A LW post that leans so heavily on the disease metaphor, and applies it to other people's beliefs and behaviors, would probably be more useful if it engaged with Yvain's argument that the disease metaphor is a political and ethical argument dressed up in medical attire.

Comment author: JGWeissman 26 April 2012 05:21:19PM 4 points [-]

I don't see where Johnicholas uses classification as a disease as a proxy for deciding whether we ought to sympathize with a person's problem or tell them to toughen up.

Comment author: fubarobfusco 29 April 2012 12:53:29AM *  -1 points [-]

I took Yvain's post as being about unpacking the idea of "disease" in general — to the point that if we know facts A, B, C, D, E, F about a particular thing P, there doesn't remain a question to be asked about whether P "is" a disease.

I'd like to see the same unpacking applied to the idea of "memetic disease" or "memetic infection" used in this post.

Comment author: JGWeissman 29 April 2012 01:16:31AM 4 points [-]

It is good to unpack what we mean by a word when the word confuses us, when despite knowing all the relevant information about a thing, we still aren't sure if that thing is in the described category. Do you have an example of such a thing that may or may not be a memetic infection or disease? If not, asking to unpack in particular the meaning of "disease" as used technically in this post, just because the word "disease" as commonly used has been excellently unpacked elsewhere, strikes me as cargo cult rationality.

Comment author: timtyler 30 April 2012 01:09:11AM *  0 points [-]

Do you have an example of such a thing that may or may not be a memetic infection or disease?

Religion is sometimes considered to be an example of a debated case. Dawkins claims it is a "virus of the mind" - while others emphasise religion's adaptive aspects. Of course, nobody (in the know) seriously doubts whether it is largely "memetic", but whether it is "parasitic" is a debated issue.

For a case where the issue is whether someting is memetic or genetic, consider adaptions to language. It is often not clear whether the fit between humans and language is more the result of organic adaptations or cultural adaptations.

Comment author: JGWeissman 30 April 2012 01:25:05AM 1 point [-]

Sure, people may argue about whether or not religion is harmful, but that is not the same as agreeing about whether it is harmful and still arguing about whether it is a "disease".

Comment author: timtyler 30 April 2012 10:59:00PM *  1 point [-]

Harmful human cultural practices are classified as being parasites, competitors or amensalists in memetics. However there are hardly any competitors or amensalists hosted by humans - since most cultural practices need to benefit from their hosts by draining them of resources in order to exist - except for a few memetic hitchhikers.

I think if you deny deleterious cultural practices can usefully be classified as a form of parasitism then you must do so on the basis of some kind of criticism of the memetic framework. As far as I know the technical criticisms of memetics are all bogus. So: the most critics can say is that they don't like the idea - or that they find it distasteful.

Comment author: timtyler 27 April 2012 10:03:05PM *  0 points [-]

I don't think Yvain was arguing against the existence of social and cultural diseases. There's no argument I can see to that effect. A good job - social and cultural "diseases" are just fine.

Comment author: gdville 05 September 2017 07:37:59PM *  0 points [-]

Realizing that this thread is a few years old, I think it is perhaps even more relevant today (with the rise of fake news, etc) than when it was first drafted.

There could be more discussion on symbiosis versus parasitism. I think the biological analogy is great, but need to differentiate the effects on the individual/cell vs. those on the organism/society. Just as with genes, I suspect that the individual/cell success is less important to meme propagation than the success of the organism/society.

Likewise, perhaps some discussion is needed with regards to how we might design/control/counter memes at the society level.

Is it feasible to do this with human level intelligence, or would it take AI superintelligence?

The possibly irrelevant rant is.

Comment author: [deleted] 30 April 2012 03:33:46AM *  0 points [-]

Since every meme must have good and bad effects, why is one meme a disease? This will depend partly on how you analyze "disease." Depending on that analysis, it might depend on some qualitative difference. One qualitative difference is that some memes keep you from criticizing it. For instance, a religious meme teaches that salvation comes from good thoughts, including belief in the deity, will substantially prevent questioning the deity. In light of recent work in social psychology on the "Spinozan" theory of understanding, understanding doctrine requires tentatively believing it. This puts understanding criticism of the religion meme outside of the believer's scope. I develop this view and apply it to both religion and morality in Unraveling the mystery of morality: The unity of comprehension and belief explains moralism and faith

Comment author: timtyler 30 April 2012 11:19:24PM 1 point [-]

Since every meme must have good and bad effects, why is one meme a disease? This will depend partly on how you analyze "disease."

Traditionally, whether a symbiote is a disease or not to a biologist is down to whether it has a negative impact on its host's fitness.

Comment author: Bruno_Coelho 02 May 2012 03:23:42AM -2 points [-]

In the cultural/nurture debate someone will bring the concept of 'meme', and they will not use strong evidences in support. New terms don't make explanations automatic more cleary.

Comment author: timtyler 26 October 2013 12:02:11PM 0 points [-]

Taking background knowledge for granted happens in many domains.

The usefulness of the term "meme" is probably best attested to by its popularity - with 250+ million references on the internet.