TL;DR I try to distinguish between what I call ‘anthropomorphic’ vs ‘biological’ reasoning and look at different ways in which they are used.
Note: This post is a small part of a larger set of ideas that makes more specific/falsifiable predictions which I hope to soon compile into a bigger post. I don't claim the ideas here are super original, but I haven't seen this distinction discussed much at depth re: current model behavior.
A large debate when it comes to talking about AI behavior seems to be that of using or not using ‘anthropomorphic’ ideas to reason about models. Roughly speaking, the argument for seems to be some combination of ‘models are trained on human data’ and ‘models act human in a lot of nontrivial ways,’ and the argument against is more ‘on principle, assuming human priors is a strong assumption’ and ‘models also have a lot of more alien behavioral qualities.’ However, I think that this idea of ‘anthropomorphic vs not’ is a false dichotomy, and, as I’ll argue in this post, we should be thinking about two separate sources of human behavioral analogies.
Specifically, I’d like to distinguish between ‘anthropomorphic’ reasoning and what I will call here ‘biological’ reasoning. Biological reasoning means here that instead of using human analogies due to pretraining data, we use human analogies because humans and models are both intelligent creatures trained with some notion of reward or evolution, which can provide them with similar ‘basic objects.’ Things coming from biological reasoning should also apply to organisms at other levels, like animals or cells. In addition, most behavioral qualities that we use biological reasoning to think about are best thought of as coming from RL (although I don’t think RL is fully necessary to install them, but this is a topic for another post). In contrast, anthropomorphic reasoning is the more typical definition, coming from the fact that models are trained on human priors, and thus using human-level objects to reason about them.
In other words, using biological reasoning to draw analogies between humans and models looks something like: “Humans and models are both organisms developed in similar ‘training’ procedures -> Organisms tend to have this quality -> This human quality can also be used to think about models”
Whereas anthropomorphic reasoning looks more like: “Models are trained on human priors -> Humans have this quality -> This human quality can be used to think about models”
To illustrate this point, I’ll list a short series of qualities and whether I think they’re biological or anthropomorphic:
Emotions:
In my opinion, emotions fall into the ‘biological’ category, as they are qualities that can be ascribed to anything that has some sort of posterior or goal, as well as some system of updating over the world, and needs to make broader changes to behavior to account for this. For example, ‘happiness/sadness’ can be thought of as certainty in some posterior. I also generally think emotions are quite nice because, unlike some other qualities, each individual emotion (up to the top ~15 most common) can be thought of potentially as very primitive, rather than the space of all emotions being primitive. Our semantic concept of these emotions is likely quite close to the true biological primitives, which makes them quite useful objects to reason with.
Personas:
From most of the common definitions (e.x. PSM), personas fall into the ‘anthropomorphic’ category. Individual personas are quite literally human characters, and even if personas have some ‘basis set,’ I also think that this basis set is largely derived from human priors. Under many definitions, the idea of ‘a persona’ is also anthropomorphic, although some definitions (like resolution’s) may lean more biological, which I discuss more below.
Agent Swarms/Collective Behaviors:
Swarms and collective behaviors overall seem quite biological to me given an organism that is trained/evolved around copies of itself or similar entities. While priors exist for swarm behaviors, they’re unlikely to produce things like we saw in something like the HuggingFace incident. Still, there are human analogies we can apply to HuggingFace incident behavior, and so I argue these come more from swarms as a shared ‘biological’ prior.
General usefulness and use as a constructive tool
I think this distinction is useful in a few ways. The first is that biological reasoning-derived qualities are more likely to be ‘true’ primitives of model behavior (note this does not necessarily imply things like the representations we find of them are true, e.x. Emotion vectors extracted from off-policy data). For similar reasons, these qualities are more likely to generalize to different model scales, architectures, and training setups. This also makes them potentially more scientifically useful: it’s likely much easier to build theory and to find basic structures in terms of the model that correspond to biological primitives rather than anthropomorphic ones, and similarly it’s much easier to make falsifiable predictions.
Building off of all of these reasons, I think that biological reasoning is very useful as a constructive tool as well. I.e., attempting to construct some quality we care about from a biological perspective likely will give something closer to the true ‘basic object.’ Perhaps a nice example here is personas - one way to define personas is from human priors. However, another way to define personas, and the way that people seem to want to use them, is some very low-rank description of behavior that comes from some more primitive basis set - something like ‘goal-seeky’ could be an example of this. Another way could be to think about constructing ‘a persona’ as a basic, organism-dependent object that means ‘a set of high-level behavioral profiles of this organism.’ Given that we’ve reached these two differing definitions, we can now consider separating them out into their own differing objects.
Does this mean ‘anthropomorphic’ reasoning is useless?
I don’t think this means anthropomorphic reasoning is useless in any sense. The fact that models have pretraining priors on human behavior is useful and should be thought about as a tool. However, I think that instead of starting from anthropomorphic reasoning, perhaps a better route would be to start from biological reasoning, and try to understand how the anthropomorphic qualities connect back up at some level. This is the subject of a larger post I’ll write at a later time.
Postscript: Other categorizations / constructions
One potential argument is that another way to get to similar things as biological reasoning is through ‘agentic’ reasoning in a more game theory-esque sense. E.x., something like a goal could be well defined through agentic reasoning. But I think using agentic reasoning to think about high-level model behavior black-boxes too much of the ‘gears-level’ idea of what’s really going on. Emotions from above provide a nice example of this: it’s a bit more difficult to get to emotions by reasoning about the model as some game-theoretic agent. However, I do think doing this game-theoretic reasoning at a more basic level, thinking about even the small internal parts of the model as separate agents, and building from there, is probably quite useful, and is likely equivalent to biological reasoning at some high level.
Another argument here is that certain human-like qualities are also primitives as you go up some ‘hierarchy’ of biological reasoning. That is, there is some number that represents ‘how far up’ you are in some hierarchy or organisms, and once you get to a certain level, certain things become basic objects. E.x. cells -> organisms -> societies, and then qualities derived from human society come from biological reasoning. This is a pretty good argument, and I’m torn on how to think about this. But, I think that there are some things that are almost definitely not primitives (E.x. ‘liking hamburgers’), and since the ‘basic features’ associated with LLMs are different from ours, it’s more unlikely that their versions of these ‘high up’ primitives correspond to our own. It’s also probably not good to assume this sort of thing without going through the exercise of trying to construct it, which is probably hard to do robustly right now.
TL;DR I try to distinguish between what I call ‘anthropomorphic’ vs ‘biological’ reasoning and look at different ways in which they are used.
Note: This post is a small part of a larger set of ideas that makes more specific/falsifiable predictions which I hope to soon compile into a bigger post. I don't claim the ideas here are super original, but I haven't seen this distinction discussed much at depth re: current model behavior.
A large debate when it comes to talking about AI behavior seems to be that of using or not using ‘anthropomorphic’ ideas to reason about models. Roughly speaking, the argument for seems to be some combination of ‘models are trained on human data’ and ‘models act human in a lot of nontrivial ways,’ and the argument against is more ‘on principle, assuming human priors is a strong assumption’ and ‘models also have a lot of more alien behavioral qualities.’ However, I think that this idea of ‘anthropomorphic vs not’ is a false dichotomy, and, as I’ll argue in this post, we should be thinking about two separate sources of human behavioral analogies.
Specifically, I’d like to distinguish between ‘anthropomorphic’ reasoning and what I will call here ‘biological’ reasoning. Biological reasoning means here that instead of using human analogies due to pretraining data, we use human analogies because humans and models are both intelligent creatures trained with some notion of reward or evolution, which can provide them with similar ‘basic objects.’ Things coming from biological reasoning should also apply to organisms at other levels, like animals or cells. In addition, most behavioral qualities that we use biological reasoning to think about are best thought of as coming from RL (although I don’t think RL is fully necessary to install them, but this is a topic for another post). In contrast, anthropomorphic reasoning is the more typical definition, coming from the fact that models are trained on human priors, and thus using human-level objects to reason about them.
In other words, using biological reasoning to draw analogies between humans and models looks something like: “Humans and models are both organisms developed in similar ‘training’ procedures -> Organisms tend to have this quality -> This human quality can also be used to think about models”
Whereas anthropomorphic reasoning looks more like: “Models are trained on human priors -> Humans have this quality -> This human quality can be used to think about models”
To illustrate this point, I’ll list a short series of qualities and whether I think they’re biological or anthropomorphic:
Emotions:
In my opinion, emotions fall into the ‘biological’ category, as they are qualities that can be ascribed to anything that has some sort of posterior or goal, as well as some system of updating over the world, and needs to make broader changes to behavior to account for this. For example, ‘happiness/sadness’ can be thought of as certainty in some posterior. I also generally think emotions are quite nice because, unlike some other qualities, each individual emotion (up to the top ~15 most common) can be thought of potentially as very primitive, rather than the space of all emotions being primitive. Our semantic concept of these emotions is likely quite close to the true biological primitives, which makes them quite useful objects to reason with.
Personas:
From most of the common definitions (e.x. PSM), personas fall into the ‘anthropomorphic’ category. Individual personas are quite literally human characters, and even if personas have some ‘basis set,’ I also think that this basis set is largely derived from human priors. Under many definitions, the idea of ‘a persona’ is also anthropomorphic, although some definitions (like resolution’s) may lean more biological, which I discuss more below.
Agent Swarms/Collective Behaviors:
Swarms and collective behaviors overall seem quite biological to me given an organism that is trained/evolved around copies of itself or similar entities. While priors exist for swarm behaviors, they’re unlikely to produce things like we saw in something like the HuggingFace incident. Still, there are human analogies we can apply to HuggingFace incident behavior, and so I argue these come more from swarms as a shared ‘biological’ prior.
General usefulness and use as a constructive tool
I think this distinction is useful in a few ways. The first is that biological reasoning-derived qualities are more likely to be ‘true’ primitives of model behavior (note this does not necessarily imply things like the representations we find of them are true, e.x. Emotion vectors extracted from off-policy data). For similar reasons, these qualities are more likely to generalize to different model scales, architectures, and training setups. This also makes them potentially more scientifically useful: it’s likely much easier to build theory and to find basic structures in terms of the model that correspond to biological primitives rather than anthropomorphic ones, and similarly it’s much easier to make falsifiable predictions.
Building off of all of these reasons, I think that biological reasoning is very useful as a constructive tool as well. I.e., attempting to construct some quality we care about from a biological perspective likely will give something closer to the true ‘basic object.’ Perhaps a nice example here is personas - one way to define personas is from human priors. However, another way to define personas, and the way that people seem to want to use them, is some very low-rank description of behavior that comes from some more primitive basis set - something like ‘goal-seeky’ could be an example of this. Another way could be to think about constructing ‘a persona’ as a basic, organism-dependent object that means ‘a set of high-level behavioral profiles of this organism.’ Given that we’ve reached these two differing definitions, we can now consider separating them out into their own differing objects.
Does this mean ‘anthropomorphic’ reasoning is useless?
I don’t think this means anthropomorphic reasoning is useless in any sense. The fact that models have pretraining priors on human behavior is useful and should be thought about as a tool. However, I think that instead of starting from anthropomorphic reasoning, perhaps a better route would be to start from biological reasoning, and try to understand how the anthropomorphic qualities connect back up at some level. This is the subject of a larger post I’ll write at a later time.
Postscript: Other categorizations / constructions
One potential argument is that another way to get to similar things as biological reasoning is through ‘agentic’ reasoning in a more game theory-esque sense. E.x., something like a goal could be well defined through agentic reasoning. But I think using agentic reasoning to think about high-level model behavior black-boxes too much of the ‘gears-level’ idea of what’s really going on. Emotions from above provide a nice example of this: it’s a bit more difficult to get to emotions by reasoning about the model as some game-theoretic agent. However, I do think doing this game-theoretic reasoning at a more basic level, thinking about even the small internal parts of the model as separate agents, and building from there, is probably quite useful, and is likely equivalent to biological reasoning at some high level.
Another argument here is that certain human-like qualities are also primitives as you go up some ‘hierarchy’ of biological reasoning. That is, there is some number that represents ‘how far up’ you are in some hierarchy or organisms, and once you get to a certain level, certain things become basic objects. E.x. cells -> organisms -> societies, and then qualities derived from human society come from biological reasoning. This is a pretty good argument, and I’m torn on how to think about this. But, I think that there are some things that are almost definitely not primitives (E.x. ‘liking hamburgers’), and since the ‘basic features’ associated with LLMs are different from ours, it’s more unlikely that their versions of these ‘high up’ primitives correspond to our own. It’s also probably not good to assume this sort of thing without going through the exercise of trying to construct it, which is probably hard to do robustly right now.