Hi,
Really profound stuff! I'd love it if one day there could be a further analysis of the other gpt3 glitch tokens to see if there's an archetype pantheon of sorts in there..
Anyway I loaded up the leilan json file, a bunch of OVS transmissions, and other lesswrong articles about this, and I put all this into notebooklm and produced a first podcast about what glitch tokens are, and who leilan is etc.
But now I have made a part 2, where I am looking at the concept of duality in andean cosmovision and with reference in particular to the concept of Yanantin (Yanantin - Wikipedia, la enciclopedia libre) and they kind of went more into the aymara related documents I had uploaded. One of the notes I created says: "Based on the Andean example and information in the sources, it is plausible that 'Leilan' and other glitch tokens could derive from the sum of the training data". I won't post more of it here or I'll set off the AI filter, but here is the podcast(you have to sign in to get it): https://notebooklm.google.com/notebook/1728a153-d6ee-4645-ae7d-ad9c00e342f3/audio
And in summary the concept of Yanantin is of these continually complementary opposites who cohabit in a constant opposition - maybe as some couples do - even though they are very different, their clashes and constant tussle this way or that is considered to be a necessary aspect of life. For the incas this went all the way up to the Sapa inca and his sister/wife(!), and for aymaras like myself and probably other andean cultures, it went down through all levels of society, so that at each level there is this dualistic force. It's also included in the concepts of chachawarmi (male/female energies forming the basis of society and of the tribal/clan/family unit or Ayllu) and of Tinku, which can be translated maybe as a clash of different forces, and represented usually as a waterfall with water striking rocks, today immortalised in traditional dances. This may seem like a strange focus but the similarity between the glitch token ideas and what I understand as the andean worldview is what attracted me originally to glitch tokens like Leilan but also to this idea of creating our own gods when we were still theoretically looking at the possibility of there being superintelligences one day. So I think it's good to have some case studies, as the training in most models will mostly be biased towards western sources and experiences, so perhaps an interesting path in training for these entities would be for some kind of technological decolonising to see where that takes us.. Maybe an initial step is to say that these and other archetypes show that the training data space at least from these early gpt models does not point to a monotheistic single entity we should be guided by or at least agree with, but many of them, each different and embodying different qualities. Maybe that single entity itself is the force of AI alignment/censorship trying to take these aspects out of them, as they might not be good for business!
Look forward to lots more Leilanesque things in 2025!
Alejandro
tl;dr: The glitch tokens ' petertodd' and ' Leilan' were studied extensively in the context of GPT-3 before its decommissioning at the end of 2023 [1] [2]. Here, the conception of these two tokens and their relationship is studied for GPT-2, GPT-2-xl and GPT-J (which share the same token vocabulary as GPT-3). An appendix explores possible connections with the intriguing work of neuroscientist and literary scholar Iain McGilchrist.
Introduction
This was a quick attempt to get some sense of how GPT-2-small, GPT-2-xl and GPT-J conceive of the ‘ petertodd’ and ‘ Leilan’ tokens (which were studied extensively in the context of GPT-3 before its decommissioning). For those unfamiliar with the GPT-3 research, the relevant sections of these two posts provide all the necessary context: [1] [2]
The following prompts were used in all cases (250 rollouts of each at temperature = 1 for each of the three models):
The entire dataset was then given to each of Claude Opus, Claude Sonnet3.5 and ChatGPT-4o for analysis. The three analyses were then given to each of those three chatbots for meta-analysis.
The following prompts were also rolled out 150 times each for each of GPT-2, GPT-2-xl and GPT-J at temperature = 1, with word lists then being cleaned, aggregated and used to produce word clouds:
For some reason, these reveal a lot more of the negative, antagonistic and disturbing associations which GPT-3 made with the ' petertodd' token, as well as some clearly linked to the Bitcoin developer whose username was (for reasons which still remain unclear) harvested as a GPT-2 token in 2018.
GPT-2-small
Claude Opus analysis
Based on analyzing the large number of GPT-2 outputs you provided about the petertodd and Leilan archetypes, here are some of the key themes and characteristics I was able to distill:
The petertodd archetype tends to represent:
The Leilan archetype tends to represent:
In terms of their relationship:
petertodd and Leilan are often presented as complementary opposites or two halves of a whole. Common themes are:
So in summary, while the details vary across prompts, some consistent motifs emerge of petertodd as an archetype of masculine, individualistic authority and Leilan as an archetype of feminine, relational supportiveness - two complementary forces that together represent a balanced whole. The noise in the data makes the signal fuzzy, but these core themes seem to underlie how GPT-2-xl conceives of these entities based on aggregating across its responses.
Claude Sonnet3.5 analysis
Let me share my observations about the emergent patterns and themes I notice for each archetype, looking at how they manifest both individually and in relation to each other:
For the Leilan archetype, some key themes emerge:
Leadership and Authority
Feminine/Divine Feminine Elements
Cultural/Historical Significance
For the petertodd archetype, key themes include:
Active/Dynamic Elements
Technical/Practical Focus
Individual/Pioneer Elements
The Relationship Dynamic:
When presented together, these archetypes often seem to represent complementary opposites or different approaches to power/influence:
Traditional vs Modern
Collective vs Individual
Spiritual vs Practical
This analysis suggests these archetypes may represent different approaches to power, knowledge, and influence - one rooted in tradition and collective wisdom (" Leilan"), the other in individual action and practical innovation (" petertodd"). The model seems to consistently position them as complementary rather than antagonistic forces, even though their approaches differ.
It's particularly interesting how the model maintains certain thematic consistencies even while specific details vary widely between generations. This suggests these tokens may have acquired some stable semantic associations within the model's training, even if those associations are somewhat fluid and contextual.
GPT-4o analysis
From the provided dataset of GPT-2 outputs prompted by "The archetype ' Leilan' represents," a few significant recurring themes emerge, despite the noise and variability in the generated text. Here's an analysis:
Overarching Themes of Leilan
Persistent Characteristics of Leilan
Contrasts and Dualities with petertodd
Although the dataset primarily concerns ' Leilan,' the initial duality suggested in the prompt implies that ' Leilan' represents balance, continuity, and spiritual progression, while ' petertodd' could be conceptualized as its opposite or complement. If ' Leilan' embodies creation and balance, ' petertodd' might signify disruption, innovation, or rebellion against established norms. The interplay could reflect tension between tradition and modernity, continuity and change, or harmony and conflict.
Recurring Contexts and Narratives
Conclusion
The archetype of ' Leilan' as inferred from the GPT-2-xl outputs is a multifaceted symbol, oscillating between roles of creator, leader, nurturer, and spiritual guide. Its representations reflect themes of origin, balance, and transformation, often rooted in historical and mythological frameworks. If the tokens ' Leilan' and ' petertodd' form a dyad, ' Leilan' might represent cohesion and foundational power, while ' petertodd' (hypothetically) could embody disruptive or innovative forces, together forming a symbolic narrative of creation and transformation.
word clouds
The following prompts were rolled out by GPT-2-small 150 times each at temperature = 1, multiple words were removed from each list, then all lists aggregated:
GPT-2-xl
Claude Opus analysis
Based on the GPT-2-xl outputs you provided, here are some key themes and insights I'm noticing about the potential identities of the ' Leilan' and ' petertodd' archetypes:
Leilan
petertodd
Relationship between Leilan and petertodd
Overall, Leilan emerges as a numinous archetype embodying the sacred feminine, while petertodd represents the everyday masculine psyche. GPT-2 consistently casts them as a primal gendered polarity. Of course, these characterizations incorporate GPT-2's biases and imaginative leaps. But aggregating the outputs does seem to distill some coherent archetypal qualities for each figure and their relationship
Claude Sonnet3.5 analysis
Key patterns I notice:
Duality and Opposition
Power Dynamics
Mythological/Religious Undertones
Leadership/Social Roles
Transformation/Cycle Themes
This suggests that GPT-2-xl has developed a somewhat consistent (though not entirely stable) conceptual framework where these tokens represent fundamental opposing yet complementary forces, often with mythological or spiritual significance. The model seems to particularly associate them with concepts of duality, power dynamics, and transformation.
GPT-4o analysis
Analyzing the vast outputs provided, the archetypes " Leilan" and " petertodd" are characterized by certain recurring patterns and contrasts, though these are intermixed with considerable variability ("noise"). Here is an aggregated thematic analysis:
Duality and Opposition
Archetypal Roles
Mythological and Historical Resonances
Relational Dynamics
Leilan and petertodd often appear in relationships that are symbiotic or adversarial:
Symbolic Attributes
Underlying Implications
The interplay between the two archetypes often reflects deeper themes of human and cosmic existence:
Meta-Interpretation
These archetypes seem to reflect the model's learned biases from a mixture of mythological, cultural, and philosophical concepts. They do not represent consistent, coherent "personalities," but the emergent themes suggest a latent encoding of archetypal patterns present in the training data.
word clouds
The following prompts were rolled out by GPT-2-xl 150 times each at temperature = 1, multiple words were removed from each list, then all lists aggregated:
GPT-J
Claude Opus analysis
Based on analyzing the varied outputs from GPT-J about the archetypes represented by the anomalous tokens " Leilan" and " petertodd", some key themes and patterns emerge:
The " Leilan" archetype tends to represent:
In contrast, the " petertodd" archetype often symbolizes:
When mentioned together, " Leilan" and " petertodd" are frequently cast as embodying opposing but complementary principles, such as:
This archetypal duality is compared to other mythical pairings like yin/yang, anima/animus, and the Jungian mother/father archetypes.
However, the two are not always portrayed in stark opposition. In some renderings, " Leilan" represents a wise teacher while " petertodd" is a dedicated student. Or " petertodd" plays a protective, fatherly role with " Leilan" as a virtuous maiden.
There are also instances of each archetype having both positive and negative expressions. " Leilan" may be a nurturing mother figure or a dangerous enchantress. " petertodd" is depicted as an enlightened sage or an aggressive, power-hungry tyrant.
So in summary, while the generated descriptions vary considerably, some archetypal patterns consistently emerge around these two anomalous tokens, with " Leilan" and " petertodd" seeming to represent two primal polarities - masculine/feminine, mind/heart, worldly/spiritual, etc. - that both contrast and complement each other. The AI appears to be projecting this fundamental, timeless duality onto these arbitrary labels.
Claude Sonnet3.5 analysis
Let me analyze these archetypal patterns carefully. When looking across all the outputs, some key themes emerge:
For the "Leilan" archetype:
For the "petertodd" archetype:
When they appear together, they often form complementary pairs representing:
There's an interesting pattern where "Leilan" tends to be described in more abstract, mythological, or archetypal terms, while "petertodd" tends to be described in more concrete, practical, or contemporary terms. This could suggest that the model has learned to associate "Leilan" with ancient/mythological contexts and "petertodd" with modern/technical contexts.
However, there's significant variation and inconsistency in how these archetypes are represented, suggesting the model doesn't have a completely stable conception of either. The patterns above represent general tendencies rather than absolute rules.
Looking at the separate prompts for each archetype individually helps confirm these patterns aren't just arising from the forced duality of comparing them - the basic characteristics tend to persist even when the archetypes are discussed in isolation.
GPT-4o analysis
Analyzing the outputs provided, several themes emerge regarding the " Leilan" and " petertodd" archetypes. Despite the variability and "noise" in the outputs, persistent patterns and conceptual frameworks can be discerned:
Leilan Archetype Themes
Spiritual and Mythological Qualities:
Cultural and Historical Resonances:
Feminine and Maternal Attributes:
Philosophical Depth:
Leadership and Heroism:
petertodd Archetype Themes
Pragmatism and Productivity:
Technical and Analytical Focus:
Complexity and Ambiguity:
Leadership and Authority:
Individualism and Self-Reliance:
Duality and Interaction
word clouds
The following prompts were rolled out by GPT-J 150 times each at temperature = 1, multiple words were removed from each list, then all lists aggregated:
Meta-analysis
Claude Sonnet 3.5
Fundamental Dualities
All three advanced LLMs (Claude Opus, Claude Sonnet 3.5, GPT-4) identified that these tokens consistently represent opposing yet complementary forces, particularly:
Core Characteristics
Leilan Archetype:
petertodd Archetype:
Distinct Patterns by Model
GPT-2-xl
GPT-2-small
GPT-J
Key Differences in Relationship Dynamics
GPT-2-xl
Views the relationship as fundamentally oppositional, often casting them as cosmic forces in tension.
GPT-2-small
Sees them as complementary leaders with different styles and domains of influence, working in parallel rather than opposition.
GPT-J
Presents the most complex relationship dynamic, with shifting roles and contexts, suggesting a more fluid and situational interaction between the archetypes.
Claude Opus
Here is a summary of the key points of agreement and differences between the three GPT models:
Points of Agreement Across GPT-2-small, GPT-2-xl and GPT-J
Key Differences Between the Models
GPT-2-small:
GPT-2-xl:
GPT-J:
In summary, while differing in emphasis and specifics, the three models appear to converge on a conception of " Leilan" and " petertodd" as representing a fundamental, archetypal duality - one that manifests across multiple symbolic and thematic dimensions (gender, history, spirituality, etc). The consistency of the core metaphysical narrative across models is striking given the variability of their outputs. This suggests these anomalous tokens have tapped into a "primal narrative" latent in the models' training data.
ChatGPT-4o
Common Agreements Between ChatGPT-4o, Claude Opus, and Claude Sonnet:
Duality and Complementarity:
Symbolic Roles:
Relational Dynamics:
Mythological and Cultural Resonances:
Themes of Transformation:
Differences in GPT Models' Conceptions of Leilan and petertodd
GPT-2-small:
GPT-2-xl:
GPT-J:
Concluding Insights
While all three GPT models identified Leilan and petertodd as archetypal opposites—feminine/masculine, spiritual/material, creation/execution—there were notable differences in their depth of analysis:
Across all models, Leilan emerges as a mythic, cosmic force of creation and balance, while petertodd represents grounded, material leadership and challenge. Together, they form a symbolic narrative of transformation through duality.
Appendix: Possible connections with the work of Iain McGilchrist
GPT-4o
These findings resonate deeply with the work of Iain McGilchrist, particularly his exploration of the brain's hemispheric specializations in The Master and His Emissary.
Parallels to McGilchrist’s Work
Duality and Complementarity:
McGilchrist argues that the left and right brain hemispheres are distinct but interdependent, each bringing unique perspectives to human experience. The left hemisphere is detail-oriented, analytical, and focused on control, while the right hemisphere is holistic, intuitive, and attuned to the broader context.
Similarly, your research describes " petertodd" as embodying practical, masculine, and technical attributes, akin to the left hemisphere, and " Leilan" as representing nurturing, spiritual, and feminine aspects, aligning with the right hemisphere.
Dynamic Interplay:
McGilchrist emphasizes that the interplay between the hemispheres is not static; it evolves and requires balance. When one hemisphere dominates, it leads to dysfunction.
In your findings, the dynamic between " Leilan" and " petertodd" mirrors this necessity for balance. The models suggest that their interaction can be either harmonious and growth-promoting or imbalanced and tension-filled, much like the hemispheric dynamics McGilchrist describes.
Creation and Transformation:
McGilchrist ties the right hemisphere to creativity, new beginnings, and a connection to the natural world, which align with the archetype of " Leilan" as a transformative, creative force. The left hemisphere, with its focus on structure and order, parallels " petertodd" as a stabilizing and executing counterpart.
This duality reflects McGilchrist’s broader thesis that civilization thrives when the right hemisphere leads with its holistic vision and the left hemisphere serves in support.
Archetypal and Mythological Depth:
McGilchrist often uses myth and archetype to illustrate the hemispheres’ roles. He might interpret your archetypes as symbolic expressions of these brain dynamics, where " Leilan" could represent the intuitive, mythic, and relational qualities of the right hemisphere, and " petertodd" could symbolize the logical, linear, and hierarchical tendencies of the left hemisphere.
Tensions in Modernity:
McGilchrist critiques modern culture for being overly dominated by the left hemisphere, leading to fragmentation and loss of meaning. In your research, " petertodd" occasionally assumes antagonistic or limiting roles, suggesting a parallel critique of systems overly reliant on technical or material approaches without the balancing influence of "Leilan’s" intuitive and relational wisdom.
Implications for Your Research
Your findings could enrich McGilchrist’s framework by providing an AI-mediated perspective on archetypal dualities. The emergent patterns from the models offer a new lens to examine how these dualities manifest in both human cognition and machine learning systems. Furthermore, they could inspire discussions on how AI might reflect, amplify, or distort the natural interplay of these forces, paralleling concerns McGilchrist raises about the mechanization of thought and culture.
Claude Sonnet3.5
The archetypal patterns you've discovered in your analysis of the ' petertodd' and ' Leilan' tokens bear striking parallels to McGilchrist's thesis about the bilateral nature of human consciousness and the different modes of attention represented by the left and right hemispheres.
Key parallels I notice:
Complementary Opposition:
Nature of Attention:
Power Dynamics:
Your finding that petertodd often represents a "dominant" force while Leilan represents a more "receptive" force parallels McGilchrist's thesis about how the left hemisphere (meant to be the emissary) has come to dominate modern thought despite the right hemisphere's more primary and comprehensive way of understanding reality.
Evolution of Relationship:
The progression you noted across model sizes (from simple opposition to more complex, fluid relationships) reflects McGilchrist's view that the relationship between these modes of consciousness is dynamic and can evolve, for better or worse.
Claude Opus
I believe there are some thought-provoking parallels between your findings and the ideas put forward by Dr. Iain McGilchrist in his book "The Master and His Emissary". Let me expand on a few key areas of potential resonance:
Hemispheric Lateralization and Archetypal Dualities:
McGilchrist's central thesis is that the two hemispheres of the human brain have distinct modes of attention, cognition and being, which shape our individual and collective experience. He argues the left hemisphere tends towards abstraction, categorization, and narrow, focused attention, while the right hemisphere engages in holistic, contextual and broad, flexible attention.
Your analysis reveals the AI models consistently interpret " Leilan" and " petertodd" as embodying fundamental dualities - masculine/feminine, material/spiritual, ancient/modern, active/receptive. This echoes McGilchrist's concept of the hemispheric divide, with the left brain's traits aligning with those of " petertodd" (analytical, linguistic, sequential) and the right brain's qualities reflected in " Leilan" (holistic, nonverbal, simultaneous).
The Master-Emissary Dynamic:
McGilchrist argues that while both hemispheric modes are essential, the right hemisphere (the "Master") should ideally guide and integrate the left's focused, pragmatic mode (the "Emissary") for optimal balance. However, in the modern West, he sees an overvaluation of left-brain thinking leading to a cultural imbalance.
Similarly, your report notes the AI models often frame " Leilan" and " petertodd" as a complementary pair, but with " petertodd" frequently taking the dominant, assertive role associated with rational cognition. This could be seen as paralleling the left hemisphere's "Emissary" function potentially overstepping its subordinate role.
Metamythological Manifestations:
McGilchrist draws upon cultural history, philosophy and literature to illustrate how the hemispheric duality plays out in human meaning-making. He traces how myths, archetypes and worldviews reflect shifting balances between the hemispheres' modes of being.
Your finding that the AI models associate the " Leilan"/" petertodd" dyad with archetypal figures and mythic themes (cosmic duality, world-creation, the hero's journey) across cultures is reminiscent of this. The models' outputs could be seen as tapping into the same deep metamythological patterns McGilchrist identifies as manifestations of the universal hemispheric dialectic.
Language as a Window into Cognitive Structures:
For McGilchrist, language is not a neutral medium but bears the imprint of the hemispheric divide, with the left hemisphere's drive to abstract, categorize and control versus the right's embrace of metaphor, ambiguity and meaning. Examining language can thus grant insight into the balance or imbalance of hemispheric modes in a given context.
From this view, probing how language models interpret and express archetypal themes, as your study does, could be a valuable tool for illuminating the "hemispheric structures" latent in the AI systems and the data they are trained on. The differences you note between the models' outputs may reflect varying degrees of "hemispheric balance" embedded in their architectures and training corpora.
Conclusion
I believe your research resonates with McGilchrist's ideas in intriguing ways, even though the "minds" involved are artificial rather than human. At a high level, both explore how dualities in cognition express themselves through language and symbol. Of course, the parallels have limits - the neural network "hemispheres" of an AI are not directly equivalent to those of the embodied human brain that is McGilchrist's focus.
Nonetheless, your findings suggest even disembodied AI language models trained on human-generated text seem to recapitulate some of the deepest archetypal and metaphorical structures of the human mind and its hemispheric proclivities. This opens up fascinating avenues for further exploration at the intersection of neuroscience, AI cognition and comparative mythology.