Thanks for writing this! For me this post is up there in terms of posts I would send back in time to my 20 years ago self, since I think a more explicit gears-y model of teleology (which I would say this is) would have helped me a lot. I think it speaks to something that is sort of obvious, but also IME often dismissed as ~magical thinking, but in part because often when I've heard people try to explain it they have invoked magical thinking.
This was an interesting article, and I am happy to see it was finished (I was reading the incomplete version, decided to refresh the page and... presto!, the complete piece was now available).
Biology is not my forte, but I think I was able to follow the argument, although until I read the conclusion, I wasn't quite sure where this was going, in spite of the intro. Like, sure, I can see these payloads emerging as islands of stability and persistence, with a degree of independence from the particular genes implementing them (whose evolutionary imperative is to 'be fruitful and multiply and fill the earth and subdue it'). But I initially struggled to see why their persistence wasn't simply a consequence of the usual selective pressures favoring genetic replication. I think I better understand the additional value of the payload lens when you discuss how established payloads can reshape the evolutionary landscape, making other adaptations more advantageous or easier to discover, particularly through the connections between evolutionary islands and the Baldwin effect.
I feel like any model which leads us away from the 'pure optimizer with a fixed utility function' is likely to reflect deeper reality better, at least at the human level, and that the payload model looks much more desirable given what we would likely agree to want, i.e., adaptive structures that can positively benefit from other structure's autonomy, where this can be taken to the next macro level, like human interactions, and that this makes cooperation and non-domination potentially instrumentally advantageous, rather than requiring overriding moral commandments (which I personally am skeptic of anyway). I still fail to see why mutual dependence would necessarily imply mutual respect or autonomy. like, A can benefit from B's independent functioning in some circumstances, but in others A might benefit more by controlling, modifying, exploiting, or eliminating B. So I'd like to ask: what does the payload lens tell us about when cooperation and respect for autonomy will emerge, beyond what we could already predict from conventional evolutionary accounts of cooperation and conflict?
what does the payload lens tell us about when cooperation and respect for autonomy will emerge, beyond what we could already predict from conventional evolutionary accounts of cooperation and conflict?
Conventional game theory about cooperation and conflict assumes I already know what my payoffs are, and what states-of-the-joint-gameboard will create my payoffs. Under that assumption, having more control is just good for me. (I can still cooperate with the other player, via trading actions that'll assist my goals for actions that'll assist theirs, or not-cooperate via being unable to coordinate such trades.)
In the world I'm talking about, the payloads are sort of part of each other's minds, and so one payload having more control is not necessarily "just good" for that payload. Payload A will often know things payload B doesn't about how the organism they're both inhabiting can best survive, and vice versa, such that each will propagate better if they can manage to avoid interfering with the autonomy of the other. (Eg A may know what bits of embryo development should go particular ways to create working eyesight, while B knows what bits should go particular ways to create a healthy cardiovascular system.) If we watch over evolutionary time, we should maybe see hillclimbing away from [situations where both use the same chemical signal to accomplish different things, such that new genes that help one will often harm the other] toward [situations where they have separate signals that can be tuned without interfering with each other], for example. (I can unpack that sentence into a whole section if someone wants; Gerhart and Kirschner talk about this in their books and articles as "weak regulatory linkage" and Michael Levin talks about the "meaning" of signals in molecular development; to my mind both are discussing ways mind-pieces can form something more like a mind. And conventional game theory isn't. I had meant to add sections on this in the above post, but then got lazy.)
The stuff I said here about payloads also in interactions between humans, but it's not part of standard game theory there either. For example: if I was somehow the communist dictator of the world this might well be worse for my goals than it would be for me to retain my current role, since I care some about the well-being of billions of people and projects but those billions of people have lots of relevant know-how about how to acquire their own well-being, that I don't have, and so I'd probably give dumb directions without meaning to, or at least fail to give the kind of intricate quickly-updating directions that people will manage to give themselves when there isn't a communist dictator. And me being in control would short-circuit peoples' processes of trying things and learning and rearranging resources to businesses that more know what they're doing and stuff.
Did I manage to make any sense?
Or: The payload lens on Darwin
Tl;dr: What is the most natural teleological shorthand to use, when thinking about evolution?
The “gene-centered view” of e.g. Dawkins says: genes are “trying” to increase the number of exact copies of themselves. And organisms are “vehicles” genes use to survive.
The “payload lens” (which I’ll be pitching) says: adaptive payloads (such as eyesight or fair meiosis) are “trying” to become deeper, stabler attractor basins that can team up with other adaptive payloads. And genes are “vehicles” these payloads use to survive.
Epistemic status: I haven’t done much of a lit search, so I’m claiming neither that any of this is original nor that it is vetted/established; I’m laying it out partly because I want to be able to build on it in some later essays about what we humans are (and what some possible AIs might be) other than optimizers. (However, I don’t mostly get to that here; and I hope this stuff may also be interesting in its own right.)[1]
Angles on Darwin
Darwinian evolution (as operationalized by Lewontin in his 1970 paper The Units of Selection, which helped organize the field) is the theory that intricate adaptations (such as those in Earth’s living organisms) tend to arise when a process satisfies three principles:
Lewontin stresses that these principles can apply at any of several levels, e.g. genes, cells, gametes, organisms, or populations.
My business in this essay will be to contrast two such concretizations: “genes as replicators,” which is the standard lens associated with Williams and Dawkins, and “adaptive payloads as replicators”, which I have not personally seen in the literature. I will ask how much sense each lens lets us make of the living world.
Genes as replicators
DNA strands can be copied with high fidelity, so that a given chunk of functional DNA (a “gene”) is often exactly preserved from parent to child. Such genes are a natural and common way to concretize 1-3. Under this lens, 1-3 becomes:
Following others, I will call this lens the gene-centered view. I believe this lens captures much, but far from all, of the ways 1-3 can help account for life’s variety.
Adaptive payloads as replicators
Adaptive payloads are innovations that make their organisms better at reproducing, such as eyesight, fair meiosis, or a preference for eating sugars. Adaptive payloads also constitute what Daniel Dennett calls “design work.”[2]
We can get an alternate concretization of Darwin’s 1-3 by taking adaptive payloads as our replicators. Under this lens, 1-3 becomes:
Adaptive payloads are not copied exactly, as genes are; but they nonetheless possess enough parent-child resemblance to satisfy “heredity” in Lewontin’s sense. (Being a “replicator” in this sense does not require exact copying.)
I will call this the “payload lens.”
It is well-established that the gene-centered view is useful; I will spend this essay arguing that the payload lens carries some additional value (though the reverse is also true; the genes lens is irreplaceable).[3]
Example: eyesight
Consider eyesight. More exactly, consider the progression over time that eyesight has in a particular lineage – a progression across many millions of years from the first eyespots to primitive eyes to more effective, robust eyes that work in various particular contexts.
The payload lens makes it easier to notice some facts about eyesight’s trajectory.
Eyesight is a stable attractor
Let’s jump in in the middle of the story, by attending to eyesight in a particular population of raccoons (say).
Imagine some process messes intermittently with some raccoons’ eyesight – perhaps there’s a new attractive food source that contains a vision-impairing toxin. Or perhaps this population recently began mating with a new population of quasi-raccoons that had previously been separated by a mountain range and the new gene-combinations sometimes fail to make fully functional eyes.
In this context, evolution won’t have to look through as many new mutations before it finds one that can counter the trouble (since this is a simpler change; fixing a previously-working machine is easier than building a machine from scratch). Also, existing variants that help with the new circumstance may quickly approach fixation (since a previously-working machine that is slightly broken offers strong selection gradients toward fixing the machine).
Thus, the presence of useful eyes will cause selection toward more robust eyes. Importantly, this rapid selection toward robust eyes happens mostly because eyesight is already present; animals with no eyes might evolve eyesight, but the selection would be far slower.
Eyesight is thus a stable attractor (and this is easy to describe in terms of “eyesight,” i.e. when focusing on eyesight as an adaptive payload, and hard to describe in terms of genes directly, since e.g. “restores eyesight” could happen via any of thousands of genes).
(Of course, it is stable only so long as it is useful; when e.g. naked mole-rats move into a niche without light, entropy gradually disrupts the payload. The “eyesight attractor” requires constant selection to preserve its adaptive payload.)
The eyesight of Theseus
The “ship of Theseus” is a ship from a famous thought experiment; the ship stayed arguably “itself” across a huge period of time, even as every piece of it eventually wore out and was replaced.
Eyesight is a kind of “ship of Theseus”; its “stable attractor” nature can be preserved across huge periods of evolutionary time, and despite massive changes to the genes that initially supported it.
Or, well, that’s a theory-driven approximation. To attempt empirical accuracy:
So, eyesight is a partial “ship of Theseus”, where most planks can be substituted out but Pax6 can’t (lest the whole ship lose its function) and opsins can’t (because they’re already quite optimized, and there doesn’t seem to be a hill-climbable path from there to a different and better solution).
Example: meiotic fairness as replicator
For an example of a payload that is even less tied to particular genes, consider meiotic fairness. As you may recall, meiosis is the process by which eukaryotic organisms create haploid gametes (sperm or eggs), and it is carefully “fair” in the sense that (almost) all alleles have almost exactly a 50% chance of making it into any given gamete. What keeps meiosis fair?
Well: what keeps it fair is the outcome of an ongoing “race” between cheating and cheat-prevention.
On the one hand, genes are constantly under selective pressure to find a way into a larger share of gametes. And so new “cheater” genes are continually arising. (Biologists call this cheating “meiotic drive.”)
On the other hand, all the other genes are under selective pressure to prevent “cheater” genes from being able to succeed. Why? Because “cheater” genes typically have, as their mechanism for cheating, some pattern that is quite bad for the organism’s normal gamete-formation process and so for its fertility – and thus if a gene can prevent other genes from cheating, it can prevent its organism from having serious reproductive trouble. (Some common cheater mechanisms: poison-antidote, where if an allele isn’t copied into the gamete the gamete often dies; centromeric drive, where an allele aggressively pulls itself into the forming egg and thereby causes segregation errors; overreplication, where a gene copies itself within the somatic cells also in ways that can cause cancer.)
The observed outcome of this process is that cheater genes are quite rare. And importantly for our purposes:
So, meiotic fairness is another place where the payload lens pays some rent that is not already being paid by the gene-centered view. (We have a stable attractor, and a “ship of Theseus” robustness to the swapping-out of alleles.)
How much of biological evolution makes more sense through the payload lens?
Are eyes and meiotic fairness unusual in this way, or is most biological adaptation similar? That is: do the adaptive payloads behind most biological function tend to have “their own momentum” to a greater extent than one might expect if one looked only through the gene-centered lens? By “momentum,” I mean the tendency of a payload to persist and continue to accrue more design work, more function, more robustness-to-disruption. If many adaptive payloads have their own momentum in this way – why? And does the payload lens also shed light on other aspects of biological evolution, beyond the “many payloads are stable attractors” point?
I don’t fully know; but I’d like to lay out some considerations.
“Momentum” examples probably include most cases where multiple genes combine to make a “simple” or “form-like” effect
Payloads probably have their own momentum (ship of Theseus style, i.e. beyond the momentum possessed by the genes that make them up) in most cases where: a) the phenotype ends up somehow “simple” or “form-like” in some approximate respect; and b) this order isn’t simply inherited from physics or chemistry; it required multi-pronged evolutionary design work to create this simplicity.
Some examples of “simple”:
Why I expect momentum where I see designed simplicity:
(Related)
Pathways between adaptations
Darwin’s core achievement is to shed light on where all the design work in life comes from.
So far, we haven’t touched most of this. Even if eyes, fair meiosis, tendons, warm-blooded temperature regulation, etc. tend to gather momentum once started … how did they get started? Does finding some of these attractors make it easier to find others? Does the payload lens help explain life’s ability to locate the functional structures we see in today’s lifeforms, or does it only explain life’s ability to keep hold of payloads already found?
I suspect the payload lens helps with the finding, as well as the keeping. I will next argue for this.
Payload geometry: islands or ridge-lines?
The question, broadly, is what the actual landscape of evolutionary biology looks like through the payload lens.
Is it more like a set of isolated islands?
Or are the islands connected by traversable ridges, where some payloads make other payloads either easier to start building (e.g. eyes may make it easier to build a predator-triggered startle reflex) or more worthwhile to keep building (e.g. eyes make brains more useful)?
(The gene-centered view has its own answer here: the more genes are around, the easier it is for new adaptations to arise by duplicating an existing gene and then allowing the new copy to specialize in a new direction. Does the payload lens have anything to add?)
Payloads change which other payloads are worth it
Adaptive payloads can make other adaptive payloads more useful. For example, eyes can make brains more useful no matter what genes implement the eyes.[6] (Likewise, brains can make eyes more useful; there’s no point in gathering data you can’t interpret.)
Also, it is easier for evolution to locate structures with a bigger payoff – e.g. if the reward for better brains is high, it’ll be easier to find a path toward them, since it’ll be acceptable to do so in ways that cost more calories or temporarily disrupt other structures.
Toeholds without genes: The Baldwin effect
Often, the hardest part about evolving a given adaptive payload is the “getting started” part: once you have an adaptation (even a weak or intermittent one) there’s often a nice climbable selective gradient that can get you to better versions, but that same gradient can’t well-lead you to the thing’s start because there’s not much advantage in having the barest beginnings of some adaptations.
My claim in this section is this: the “having a toehold makes it way easier to evolve the rest of an adaptive payload” phenomenon can hold even when the initial toehold is triggered by the environment, rather than by genes!
To see this, consider a population of crows who live near walnut trees. The crows would like to eat the walnuts; and it is theoretically possible for the crows to evolve genes that predispose them to take the complex actions required to crack the walnuts (namely, to drop walnuts onto hard surfaces from very high in the air).
Absent learning, evolution would need a very long time to locate genes for such a complex behavior because its reward function is sharply all-or-nothing. (Doing part of the walnut-cracking ritual gets you zero walnuts.)
Fortunately for the crows, they can learn. And so they occasionally learn to crack the walnuts via imitation and reasoning, even without genetic help. And genetic variations that predispose them slightly toward the right walnut-cracking rituals let them succeed more often, making a nice climbable gradient toward genes that encode more and more of what they need to do. This is called the “Baldwin effect.”
The above example is theoretical, but Geoffrey Hinton and Steven Nowlan have a nice paper with simulated examples, and biologists have also demonstrated some real examples. Biologists disagree with each other about how common and significant Baldwin effect examples are, though my personal guess is “common.”
Cascades of toeholds
In math, it is frequently claimed that a student needs “mathematical maturity” before they can properly study upper division mathematics. That is: they need, not any particular knowledge, but an elusive something that comes from working through a certain amount of other math. Similarly, the fifth type of partner dance you learn will typically be much easier than the first, even when you aren’t re-using any particular dance-step patterns in a way you can point to.
More broadly, some skills make it easier to learn other skills via resembling them in some subtle fashion, even where there’s no obvious “this is a step in that” relationship.
I suspect that many payloads, in combination with many environments and in-environment learning, can similarly make it easier to get a toehold toward various new payloads. (“Learning,” here, does not have to be in the manner of an animal with a nervous system; it can also be e.g. an organism of whatever sort adapting to excess salinity using its homeostatic mechanisms, or any other within-lifetime adaptation.) Skills that are from some angles separate are from some other angles reminiscent of one another, such that there may be a graph whereby learning some payloads gives a lineage a toehold toward others, which then acquire their own momentum and give the lineage toeholds toward still others. (Examples: the chemical receptors → light-detectors example we met earlier; feathers being used initially for warmth but offering a toehold toward flight. Biologists call this “exaptation.”)
Why I care
There are several reasons I care about the payload lens, some of which I hope to reach in later posts. But I’ll touch lightly on some now.
A toy model of something corrigibility-adjacent
A payload A will often “want” another payload B to be thriving, to be accurately oriented to its surroundings, and to do what B thinks best rather than being controlled by A (in cases where B has important priors or skills that A doesn’t, which will be many cases). In less teleological language: A will often survive-and-propagate better if B can do this (and if A can avoid interfering with B’s doing this).
Thus, the payload lens offers (I think) a toy model of something corrigibility-adjacent. I’m interested in this model because I might want people to grow artificial systems that respect other people or payloads in this way, and because I would personally like to learn to relate to other people and payloads in this way. Also I’m enough of a LessWronger to want mathy models of my deep desires where I can get them, and “team up well while respecting what others are, and while having them respect me” is among my deep desires. (Though, to be clear, this is at best a hint as to how to model it. Or a “toehold,” optimistically.)
A hint at where simple structures and functions may come from
I also sketched a bit of where “simple” or “intelligible” structures within organisms might appear from. I like having toy models of this. I’m fairly sure there’s more here.
What am I? What am I for?
Am I an optimizer with a fixed utility function, wishing, if I can, to control the lightcone and transform it according to my own preexisting values? Or something else?
Personally, I like to think of myself as a payload, with my heart looking after many of the “useful, payload-y” parts of genes and memes, and with this payload having reached me via the efforts of many genetic and memetic ancestors who worked hard and choosily to figure out what parts to pass on, and what new payloads to join with as they did so.
And I’d like to think of me, this moment, my choices, as a case where many of these payloads are somehow collaborating with each other in choosing paths forward that can honor most of what I carry, and in looking for what other payloads (even of alien origin) I might wish, and be able to, team up freely with.
This is a choice on my part – I’m taking poetic inspiration from the biology I’ve sketched above; it’s not forced.
I do owe substantial intellectual debts to: Gerhart and Kirschner (whom I last read in 2007, but whose work stayed with me); Michael Levin; Daniel Dennett; and Friedrich Hayek, although in each case I got patterns of thought more than citeable specifics. I would also like to thank a Claude Fable 5.1 instance who helped me with biology facts and pointers, drew my diagrams, and helped me tighten the concepts. Remaining errors are of course my own.
"Design work" here means "stuff (such as a watch, or an intricate bit of biological function) that must be the result of some sort of "research and design"-type process, such as a human engineer, biological evolution, or memetic trial and error across a large population.
Dennett's concept originates (in some sense) in William Paley's famous thought experiment in which a man, walking across a heath, happens upon a watch and infers, as he looks at the intricate ticking gears (even without having seen a watch before), that there must somewhere be a watch-maker. Dawkins later riffed off this thought experiment by referring to evolution as a "blind watch-maker," as did Dennett with his notion of "design work," which he uses to describe anything that visibly contains R&D.
To see at once that the genes lens is great, notice e.g. that we today see no life without nucleic acid sequences – evolution has not produced lineages that can thrive and produce offspring without genes.
Although, evolution didn’t get opsins from nowhere; opsins evolved from previous sensor-proteins that cells used to detect chemicals, and the thing where a previous payload (G-protein-coupled receptors) made it easier for this new payload (vision) to arise… does remind me of the payload lens. See the section Cascades of toeholds.
Apparently biologists have a term for a related sort of substitution – “developmental system drift,” as introduced by True and Haag, 2001; the authors’ claim, like mine, is that sometimes selection pressures cause aspects of a phenotype to remain stable despite substantial changes in genes. (Unlike me, the authors know lots of biology and can root their generalizations in empirics.)
Except insofar as they change the eyes-brains interface