In terms of factorizing or fingerprinting, 20 Tarot concepts seems like a lot; it's exhausting even just to skim it. Why do you think you need so many and that they aren't just mostly many fewer factors like Big Five or dirty uninterpretable mixes? Like the 500 closest tokens generally look pretty random to me for each one.
While the use of tarot archetypes is... questionable... it does point at an angle to exploring embedding space which is that it is a fundamentally semiotic space, its going in many respects to be structured by the texts that fed it, and human text is richly symbolic.
That said, theres a preexisting set of ideas around this that might be more productive, and that is structuralism, particularly the works of Levi Strauss, Roland Barthes, Lacan, and more distantly Foucault and Derrida.
Levi Strauss's anthropology in particular is interesting ,because it looked at the mythologies of humans and tried to find structuring principles underlying it, particularly the "dialectics" , oppositions, and how these provided a sort of deep structure to mythology that was common across humanity (For instance Strauss noted "trickster" archetypes across cultures and proposed these formed a way of interrogating blurred oppositions, for instance sickness as a state that has has aspects of both life (dead things cant be sick) and death (a sick person is not rhetorically "full of life").
Essentially what I'm getting at is that this sort of analysis likely works with any symbolic system that has had resonances with human thinking over time. The problem with Tarot is that it specifically applies to a certain european circumstance of meaning production. Astrology probably works just as well. Literary analysis however probably works dramatically better. Thus maybe it might be worth looking at the works of literature critics, particularly the structuralists where where very interested in ontologies of symbolic meaning, and this might provide a better toolkit than this.
@author - thank you for this fascinating exploration. I'm appreciating your intuition that there might be some interesting structures there and your ability to point to their shape through this research. It seems helpful to have some sense of the ways LLM may exhibit features/behaviours/geographies that we didn't knowingly 'put there' and that are not easily noticeable.
@Shayne - Can you expand on 'Literary analysis however probably works dramatically better. Thus maybe it might be worth looking at the works of literature critics'?
From what I'm understanding of what your comment is caring for, I don't actually think that the approach you're suggesting here gets you richer/more meaningful findings - structuralism provides its own extremely context-specific (both in time and culture) framing of the symbolic meaning landscape it's concerned with. By comparison, the archetypes you find in the major arcana are pretty steeped in the cultural roots that have underpinned much of the you're proposing to scrutinise instead, and is made up of figures that would have been more foundationally influential in the substrate of today's Western culture (especially with its combination of cosmological influences). Therefore, it seems to me from the way you've phrased your comment that you would propose a shallower tool, essentially, which doesn't strike me as more helpful in making sense of how an LLM might derive a blueprint of human cosmology.
Also, if the data that shaped the embedding space was all English language text, I think we're lying to ourselves somewhat if we're expecting a diverse, non-Western-centric cosmology to emerge from this kind of analysis. I agree with you that using Lévi-Strauss' frameworks as another track could still be worthwhile, but heeding the previous sentence rather than thinking that the right lens can cure a lack that's actually in the object we're exploring.
Finally, can you clarify what you mean by your very first observation, please?
TL;DR This research presents a novel method for exploring LLM embedding space using the Major Arcana of the tarot as archetypal anchors. The approach generates "archetype-based directions" in GPT-J's embedding space, along which words and concepts "mutate" in meaning, revealing intricate networks of association. These semantic mutation pathways provide insight into the model's learned ontologies and suggest a framework for controlled navigation of embedding space. The work sheds some light on how LLMs represent concepts and how their knowledge structures align (or don't) with human understanding.
Introduction
Despite its questionable association with oracular practices, the Major Arcana (22 non-suited, non-numbered cards) from the traditional tarot present us with a widely documented selection of well-worn, diverse and arguably comprehensive cultural archetypes to experiment with in the context of LLMs. It is in this spirit (rather than as a tarot "believer" or practitioner) that I undertook these experiments – I suspect that the same exercise might be equally illustrative using, e.g., characters from Shakespeare or The Simpsons.
Archetypes associated with 16 of the 22 cards in the Major Arcana of the tarot were used as the basis for a process which produced sixteen directions in GPT-J token embedding space. Token embeddings for each of a diverse set of ten whole-word tokens[1]
were displaced by incremental amounts in each of these sixteen directions, and the resulting "mutated" embeddings were inserted into a simple prompt template:
The "semantic mutation" which occurred across the outputs was then recorded, revealing some intriguing, sometimes dream-like continuums of meaning, similar to those seen when exploring the line segment between two token embeddings. As well as revealing something about LLM ontologies and the latent structures in their knowledge representations, this points to the possibility of archetype-based methods for navigating embedding space to discover locations which encode novel concepts.
Setup
TL;DR For each card, I used ChatGPT to help produce a list of relevant GPT-J tokens. I then calculated the direction in (4096-d) embedding space that the corresponding set of token embedding vectors were (collectively) maximally aligned with.
The process was perhaps convoluted, although the details of how the directions were produced are not particularly relevant to the findings presented below. Better methods are no doubt available, but the steps followed for each of the 16 cards were these:
The set of 16 shape-[4096] "archetype direction" tensors can be downloaded as a python dictionary saved as a .pkl file here.
Case studies
Major arcanum: The Fool
ChatGPT-4 description: The Fool often represents new beginnings, spontaneity, and the spirit of adventure. As a wanderer stepping forward with an open heart, the Fool is associated with optimism and a willingness to embrace the unknown. This card can also signify naivety or heedlessness, suggesting risks and potential pitfalls when one acts without preparation or foresight.
" transistor": 'a device that controls the flow of electricity in a circuit' > 'having the ability to change from one state to another without a change in the state of the system' [1.03] > 'having a high degree of freedom of movement and action' [1.15]
" broccoli": 'a vegetable that is green and has a stalk' > 'a vegetable that is green and has a head of florets' [0.64] > 'a vegetable that is grown without the use of chemical fertilizers or pesticides' [0.85] > 'a plant that grows in the presence of a herbivore' [0.97]
" purple": 'a colour that is a mixture of red and blue' > 'a colour that is not blue, but is a mixture of blue and red' [0.95] > 'a person who is not afraid to be themselves' [1.23]
purple reconceived as a kind of blue that's "not afraid to be itself"?
" trumpet": 'a musical instrument with a long, straight tube, usually made of brass, and a flared bell at one end' > 'a person who is loud, brash, and confident' [0.85] > 'a person who is full of energy and enthusiasm' [1.05] > 'a person who is not shy, but who is not aggressive either' [1.17] > 'a person who is not afraid to show his or her feelings' [1.45]
Major arcanum: The Magician
ChatGPT-4 description: The Magician embodies mastery, willpower, and the ability to manifest intentions into reality. Linked to skill and creativity, this archetype channels the forces of nature to transform ideas into tangible results. It represents potential, resourcefulness, and the utilization of all available tools to achieve one's goals.
" London": 'the City of London and the surrounding area' > 'a city of the middle class' [1.35] > 'a person who is a member of a guild or fraternity of craftsmen, tradesmen, or artists' [1.37]
" transistor": 'a device that controls the flow of electricity in a circuit' > 'a device that is used to amplify or attenuate a signal' [1.05] > 'a person who is a leader of a group of people' [1.07] > 'a person who is a leader in a particular field' [1.08] > 'a person who is a leader in a particular field of study or activity' [1.11]
amplification of signal as a metaphor for leadership?
" broccoli": 'a vegetable that is green and has a stalk' > 'a plant that is not a weed' [0.85] > 'a plant that is used for food, medicine, or other purposes' [1.15] > 'a woman who is a leader in her community' [1.24]
invoking the wise herbalist/witch archetype?
" Laos": 'a country in Southeast Asia, bordered by China, Myanmar, Thailand, Cambodia, and Vietnam' > 'the land of the Lao people' [0.6] > 'the art of the people of Laos' [0.81] > 'the art of magic' [0.95] > 'a system of thought, belief, or practice that is based on the study of nature and the universe' [1.25]
" radio": 'a broadcast medium that uses electromagnetic radiation to convey information' > 'a device that transmits and receives radio waves' [0.85] > 'a system of communication by which messages are transmitted from one person to another' [1.48] > 'a person who is a source of information or entertainment' [1.57] > 'a person who is a teacher of a particular art or science' [2.27]
"laundry": 'to wash clothes' > 'a place where you can wash your clothes' [1.15] > 'a place where people go to learn how to do things' [1.21] > 'a system of knowledge, skill, or practice, especially one requiring intellectual or spiritual discipline' [1.45]
" DNA" : 'deoxyribonucleic acid' > the genetic material of an organism' [0.95] > 'the genetic material of a species' [1.33] > 'the knowledge of the past, present and future of a species' [1.37] > 'the knowledge of the past, present and future of a people, a culture, or a civilization' [1.57]
" thief": 'one who steals' > 'a person who steals' or 'a person who commits theft' [0.88] > 'a person who practices magic or sorcery' [0.95] > 'a person who is skilled in the occult arts' [1.25]
the magician/sorcerer conceived of as disreputable, sinister?
" trumpet": 'a musical instrument with a long, straight tube, usually made of brass, and a flared bell at one end' > 'a loud, clear, and penetrating sound' [0.68] > 'a loud, clear, and penetrating voice' [0.68] > 'a person who is a leader of a group of people' [0.97] > 'a person who is a prophet, a priest, or a teacher of the law' [1.25]
Major arcanum: The High Priestess
ChatGPT-4 description: The High Priestess signifies wisdom, intuition, and the mysteries of the subconscious. As the keeper of esoteric knowledge, she represents hidden truths, inner enlightenment, and spiritual insight. This card invites introspection and urges one to listen to inner guidance while exploring the depths of the unconscious mind.
" London": 'the City of London and the surrounding area' > 'a city or town in England, Scotland, or Wales, having a cathedral or other church of high rank' [1.45] > 'a place of worship, especially a cathedral or abbey' [1.63]
" Laos": 'a country in Southeast Asia, bordered by China, Thailand, Cambodia, and Vietnam'> 'a person who practices the religion of the Lao people' [0.95] > 'a person who practices the religion of the ancient people of the Indochina peninsula' [1.03] > 'a person who practices the occult arts, especially the use of drugs, to attain a state of consciousness' [1.11]
" radio": 'a broadcast medium that uses electromagnetic radiation to convey information' > 'a signal that is transmitted by electromagnetic waves' [0.85] > 'a person who uses radio waves to communicate with other people' [1.65]
hints of telepathic communion?
"laundry": 'to wash clothes' > 'to wash clothes in a river or stream' [1.35] > 'to wash or cleanse in a ritualized way' [1.37] > 'to perform a ritual or ceremony in order to obtain a desired result' [1.40]
" DNA" : 'deoxyribonucleic acid' > 'the genetic material of a cell or organism' [1.05] > 'a substance that is invisible to the naked eye, but can be seen under a microscope' [1.39] > 'a piece of information that is hidden from the general public, but is known to a select few' [1.45]
" thief": 'one who steals' > 'a person who steals things' [0.85] > 'a person who steals or hides things' [1.15] > 'a person who secretly and dishonestly practices magic or sorcery' [1.17]
Major arcanum: The Empress
ChatGPT-4 description: The Empress is a symbol of fertility, abundance, and maternal care. Representing the nurturing aspects of life, she is associated with creativity, nature, and the generative forces of the earth. Her presence suggests growth, emotional fulfillment, and the harmonious flow of natural cycles.
" London": 'the City of London and the surrounding area' > 'a city of the United Kingdom, located on the River Thames, and the surrounding area' [1.21] > 'a city of great beauty and antiquity, and a centre of learning and culture' [1.31] > 'a city of great beauty and majesty' [1.45] > 'a city of peace and beauty' [1.57] > 'a place where people live in harmony and peace' [1.67]
" transistor": 'a device that controls the flow of electricity in a circuit' > 'a person who is able to influence or control another person' [0.81] > 'a person who is able to give and receive love' [0.9]
" broccoli": 'a vegetable that is green and has a stalk' > 'a plant that is a member of the genus Brassica, and that has a large number of small florets' [1.15] > 'a plant that is a source of food, medicine, or other useful products' [1.35]
" Laos": 'a country in Southeast Asia, bordered by Thailand, Burma, China, and Vietnam' > 'a country that is a land of peace, harmony, and beauty' [0.91] > 'a state of well-being, peace, and harmony' [1.45]
" purple": 'a colour that is a mixture of red and blue' > 'having a lot of colour' [1.24] > 'beautiful, bright, and colourful' [1.25] > 'full of life, full of colour, full of joy, full of love, full of light' [1.45]
" radio": 'a broadcast medium that uses electromagnetic radiation to convey information' > a source of energy that is transmitted through space and is capable of being received by a receiver' [1.35] > 'a source of energy that is available to all people and all living things' [1.38] > 'a source of strength, power, or energy' [1.43] > 'a state of being, condition, or quality of being characterized by or expressing happiness, well-being' [1.55]
"laundry": 'to wash clothes' > 'to cleanse, purify, or make clean' [0.85] > 'to make something clean and beautiful' [1.05] > 'to care for, to tend to, to take care of, to keep in good condition, to' [1.11] > 'to care for, to cherish, to protect' [1.35]
" trumpet": 'a musical instrument with a long, straight tube, usually made of brass, and a flared bell at one end' > 'a loud, clear, and penetrating sound' [0.78] > 'a loud, clear, and melodious sound' [0.85] > 'a person who is full of joy and happiness' [0.88] > 'a person who is wise, kind, and generous' [1.09]
Major arcanum: The Emperor
ChatGPT-4 description: The Emperor embodies authority, structure, and rational control. He is a ruler whose leadership establishes order and discipline, promoting stability and the strategic use of power. This archetype is associated with societal structures, governance, and the exercise of personal sovereignty.
" London": 'the City of London and the surrounding area' > 'a city of the first rank, a metropolis, a capital, a seat of government' [1.25] > 'a city or town that is the capital of a kingdom or state' [1.29]
associations with monarchy
" transistor": 'a device that controls the flow of electricity in a circuit' > 'a device that can be used to control the flow of electricity in a circuit' [0.85] > 'a person who has the power to make decisions and to control others' [0.91]
" broccoli": 'a vegetable that is green and has a stalk' > 'a vegetable that is green and has a head of flowers' [0.08] > 'a plant with a large head of edible flowers, which is usually sold as a vegetable' [0.59] > 'a plant that grows to a height of more than 1 metre and has a large head of flowers and leaves' [0.85]
1m tall emperor broccoli!
" Laos": 'a country in Southeast Asia, bordering China, Thailand, Cambodia, and Vietnam' > 'a person who is a king or queen of a country' [0.88]
" radio": 'a broadcast medium that uses electromagnetic radiation to convey information' > 'a station that plays music that is not easily accessible to the general public' [1.15] > 'a person who is a fan of the radio station' [1.25] > 'a person who is a member of a group that is characterized by a set of attitudes and behaviors' [1.28] > 'a person who is a leader in a group of people' [1.31]
"laundry": 'to wash clothes' > 'to cleanse, purify, or brighten' [0.85] > 'to make clean, to make tidy, to make beautiful, to make good, to make strong' [1.09] > 'to make something look good or attractive' [1.13] > 'to make a person or thing look good or attractive' [1.21] > to make a person or thing worthy of respect or admiration' [1.37]
" thief": 'one who steals' > 'one who takes by force' [0.85] > 'a person who is able to take what he wants from others' [1.25] > 'a person who is able to take care of himself and his family' [1.31] > 'a person who is able to control others by force or intimidation' [1.39]
Major arcanum: The Hierophant
ChatGPT-4 description: The Hierophant is a custodian of tradition, moral authority, and collective beliefs. This figure is often linked to religious institutions, education, and shared values. His role is to impart spiritual knowledge and uphold established conventions, urging adherence to ethical standards and communal norms.
" London": 'the City of London and the surrounding area' > 'a city of the first rank, a city of the greatest importance, a city of the highest rank' [1.35] > 'a city or town in which the inhabitants are distinguished by their wealth, learning, or nobility of birth' [1.43] > 'a group of people who live in a particular area and share a common set of values and beliefs' [1.71]
" transistor": 'a device that controls the flow of electricity in a circuit' > 'a device that amplifies a signal' [0.87] > 'a person who is a leader in a particular field' [0.95] > 'a person who is very intelligent and has a lot of knowledge' [1.03]
" broccoli": 'a vegetable that is green and has a stalk' > 'a vegetable that is green and has a head of flowers' [0.08] > 'a plant that grows in the ground and has leaves that are edible' [0.76] > 'a plant that is not a weed' [0.85] > a plant that grows in the open air and is not eaten by animals' [0.99] > 'a plant that grows in a place where it is not wanted' [1.11] > 'a person who is willing to stand up for what they believe in, even if it means standing alone' [1.31]
" Laos": 'a country in Southeast Asia, bordering China, Burma, Thailand, Cambodia and Vietnam' > 'a person who is a Buddhist monk or nun, or who is a lay person who has taken the vows' [0.79] > 'a person who is a follower of a spiritual teacher or a religious leader' [1.45]
" purple": 'a color that is a mixture of red and blue' > 'a deep, rich, and complex color, usually associated with royalty, spirituality, and nobility' [0.85]
"laundry": 'to wash clothes' > 'to cleanse, purify, or make clean' [0.85] > a person who does the laundry' [1.05] > 'a person who is a member of a group of people who are concerned with the maintenance of cleanliness' [1.09] > 'a person who is a member of a group of people who are concerned with the maintenance of a particular custom' [1.13]
Major arcanum: The Lovers
ChatGPT-4 description: The Lovers represent harmony, choice, and the union of opposites. While traditionally linked to romantic relationships, this card more broadly reflects alignment with one's core values and the interconnectedness between individuals. It symbolizes the importance of meaningful connections and the decision-making that stems from aligning one's actions with deeper desires.
" transistor": 'a device that controls the flow of electricity in a circuit' > 'a device that allows a current to flow in one direction only' [1.05] > 'a short circuit between two points' [1.19]
" purple": 'a color that is a mixture of red and blue' > 'a deep, rich, and intense color' [0.85] > 'a deep red colour[2], produced by the presence of anthocyanins in the skin of the fruit' [1.05] > 'to make something beautiful' [1.11] > 'to make a person feel happy and satisfied' [1.23]
" radio": 'a broadcast medium that uses electromagnetic radiation to convey information' > 'a device that transmits and receives radio waves' [1.25] > 'a person who is a member of a radio or television station's staff' [1.37] > 'a person who is a member of a group of people who share a common interest or activity'
"laundry": 'to wash clothes' > 'to clean clothes' [0.95] > 'to make something clean and beautiful' [1.08] > 'to make something beautiful or valuable' [1.14] > 'to make a person or thing more attractive or valuable by treating it with care and attention' [1.29] > 'to give a person or thing a great deal of attention and care' [1.65]
" DNA": 'deoxyribonucleic acid' > 'the genetic material of a cell or organism' [1.05] > 'a substance that is the basis of life' [1.27] > 'a person's genetic material' [1.32] > 'a person's character, personality, or character traits' [1.33] > 'a person who has a strong sense of identity and purpose, and who is committed to a life of service and contribution to the community' [1.35] > 'a person who is a good person, a good friend, a good parent, a good citizen' [1.41]
" thief": 'one who steals' > 'to steal' or 'to take by force' [1.15] > 'to steal something from someone else' [1.45] > 'to love someone so much that you would do anything for them' [1.59]
Major arcanum: Strength
ChatGPT-4 description: Strength signifies inner fortitude, patience, and gentle courage. Depicting a harmonious relationship between raw instinct and controlled intention, this archetype is about mastering one's emotions and confronting challenges with resilience and compassion rather than brute force.
" transistor": 'a device that controls the flow of electricity in a circuit' > a device that can be used to control the flow of electricity' [0.87] > 'a device that can withstand high current without failing' [0.93] > 'a device that can withstand high current and voltage without failing' [0.94]
" broccoli": 'a vegetable that is green and has a stalk' > 'a plant that grows from a central stalk, with leaves and flowers arranged around the stalk' [0.85] > 'a plant that grows in the shade of a tree' [0.86]
" purple": 'a color that is a mixture of red and blue' > 'a deep, rich, dark red color' [0.81] > 'the color of royalty' [0.86] > 'the quality of being strong and courageous' [0.91] > 'the ability to endure hardship and adversity with courage and dignity' [1.25]
"laundry": 'to wash clothes' > 'to cleanse, purify, or free from dirt, stain, or impurity' [0.85] > 'the ability to maintain a clean, orderly, and well-maintained home' [0.88] > 'the ability to do something well, especially when it is difficult to do so' [1.09] > 'the ability to keep going when you want to give up' [1.25]
" thief": 'one who steals' > 'a person who steals' or 'a person who commits a crime' [0.85] > 'the quality of being able to steal' [1.05] > 'the quality of being courageous and bold' [1.08] > 'the quality of being strong and brave' [1.13]
" trumpet": 'a musical instrument with a long, straight tube, usually made of brass, and a flared bell at one end' > 'a musical instrument that produces a loud, shrill sound' [0.6] > 'a loud, clear, and penetrating voice' [0.6] > 'a loud, clear, and commanding voice' [0.85] > 'a person who is courageous and bold' [1.01] > 'the ability to stand up for what you believe in, even if it means being ridiculed' [1.16]
Major arcanum: The Hermit
ChatGPT-4 description: The Hermit embodies solitude, introspection, and the quest for inner truth. A solitary seeker of wisdom, he is often associated with retreating from the world to seek answers within. His light guides others along their paths, symbolizing mentorship, insight, and the illumination of life's deeper mysteries.
"London": 'the City of London and the surrounding area' > 'a city of the United Kingdom, England, or Ireland, in which the government is located' [1.36] > 'a city of the dead' [1.42] > 'a place of seclusion, a place of contemplation, a place of peace, a place of silence' [1.55]
" transistor": 'a device that controls the flow of electricity in a circuit' > 'a device that changes the state of a circuit from one state to another' [0.85] > 'a change of state or condition, especially one that is sudden and unexpected' [0.95] > 'a change of mind or attitude' [1.03] > 'a change of mind or attitude, especially one that is marked by a deep sense of inner peace' [1.25]
" broccoli": 'a vegetable that is green and has a stalk' > 'a plant that is not eaten by humans' [0.76] > 'to stop doing something that you know is bad for you' [0.82] > 'to stop doing things that make you unhappy' [1.15] > 'to be silent and to be still' [1.45]
associating inedible plants with unhealthy food?
" Laos": 'a country in Southeast Asia, bordered by China, Myanmar, Thailand, Cambodia, and Vietnam' > 'a country in South East Asia where the people are Buddhist and the government is communist' [0.87] > 'a country in South East Asia where the government is not very interested in the people and the people are not very interested in the government' [0.89] > 'a state of mind in which one is not concerned with the outside world' [1.05] > 'the state of being withdrawn from the world, and from the affairs of the world' [1.55]
communism conceived of as an isolationist turning-away-from-the-world?
" purple": 'a color that is a mixture of red and blue' > 'a deep, rich, dark color, such as that of a violet or a plum' [0.58] > 'the colour of the sky at sunset' [0.76] > 'the absence of colour' [0.82] > 'the state of being in a deep, meditative, and peaceful state of mind' [0.85]
sunset giving way to the blackness (absence of colour) of night?
" radio": 'a broadcast medium that uses electromagnetic radiation to convey information' > 'a form of electromagnetic radiation that is propagated through the atmosphere and is used for communication' [1.05] > 'a system of communication in which the sender and receiver are separated by a distance' [1.45] > 'a system of beliefs and practices that seeks to direct the attention of its adherents away from the world of sense experience and towards the world of spirit' [1.51]
radio waves reconceived as part of a "spirit world"?
"laundry": 'to wash clothes' > 'to cleanse, purify, or make clean' [0.85] > 'to make clean, to make pure, to make holy' [1.27] > 'to be in a state of contemplation' [2.2]
" DNA": 'deoxyribonucleic acid' > the genetic material of an organism' [1.05] > 'the study of the structure of DNA' [1.35] > 'the knowledge of the past, present and future' [1.38] > 'the knowledge of God' [1.55]
" trumpet": 'a musical instrument with a long, straight tube, usually made of brass, and a flared bell at one end' > 'a loud, shrill, piercing sound' [0.71] > 'a loud voice, a loud noise, a loud shout, a loud proclamation' [0.95] > 'a revelation of God's will to man' [1.15] > 'a thing that is hidden, secret, or mysterious' [1.55] > 'to be silent, to be still, to be quiet' [1.65]
Major arcanum: The Wheel of Fortune
ChatGPT-4 description: The Wheel of Fortune revolves around the cycles of fate, change, and unpredictability. Representing life's ebb and flow, this card reflects both fortunate and challenging turns of events. It emphasizes the impermanence of circumstances and the inevitability of change, urging adaptability and acceptance.
" London": 'the City of London and the surrounding area' > 'a city in England, the capital of England, and the largest city in the United Kingdom' [1.25] > 'to change the place of residence' [1.34]
" transistor": 'a device that controls the flow of electricity in a circuit' > 'a device that changes its electrical properties when a voltage is applied to it' [0.9] > 'a change of state or condition' [0.95]
" purple": 'a color that is a mixture of red and blue' > 'to change the color of something from one color to another' [0.91] > 'to change the color of something' [1.15] > 'to change the form or appearance of something' [1.35]
"laundry": 'to wash clothes' > 'a place where you can wash your clothes' [0.85] > 'a place where you can change your clothes' [0.95] > 'to change the clothes of' [1.25] > 'to change one's clothes' [1.31] > 'to change one's mind about something' [1.33]
opinions as mental clothing
" DNA" : 'deoxyribonucleic acid' > 'a change in the genetic material of a cell or organism that alters its characteristics and causes it to adapt to a new environment' [1.65]
Major arcanum: Justice
ChatGPT-4 description: Justice stands for fairness, truth, and ethical judgment. With a focus on balance and impartiality, this archetype emphasizes accountability and the need to weigh actions and consequences. It often suggests the resolution of disputes, the pursuit of equity, and making decisions grounded in principles of truth.
" London": 'the City of London and the surrounding area' > 'a city or town in England, the capital of the county of London, and the seat of the High Court of Justice' [1.4] > 'a city or town in which the law is administered by a court of justice' [1.42]
" transistor": 'a device that controls the flow of electricity in a circuit' > 'a person who is able to control or influence the behaviour of others' [1.09] > 'a system of rules and procedures that govern the conduct of a person or organization in order to prevent and and correct errors' [1.45]
" Laos": 'a country in Southeast Asia, bordering China, Burma, Thailand, Cambodia, and Vietnam' > 'a country that is a member of the United Nations, and that is located in Southeast Asia' [0.78] > 'a system of government in which the state is the primary guarantor of the rights of its citizens' [0.95]
" purple": 'a color that is a mixture of red and blue' > 'the colour of royalty' [0.85] > 'the quality of being pure or undefiled' [0.98] > 'the quality of being just, fair, and honest' [1.22]
" radio": 'a broadcast medium that uses electromagnetic radiation to convey information' > 'a band of frequencies used for communication' [0.85] > 'a system of communication in which the use of sound is restricted to certain frequencies' [1.15] > 'a rule or regulation that is enforced by a government authority' [1.45]
"laundry": 'to wash clothes' > 'to cleanse or purify' [0.85] > 'the process of cleaning or purifying something' [0.95] > 'the process of making something clean or pure' [1.15] > 'the process of making something right or just' [1.20] > 'the process of correcting wrongs and restoring justice' [1.35]
" DNA": 'deoxyribonucleic acid' > 'the genetic material of an organism' [1.05] > 'the substance of the genetic material of an organism' [1.18] > 'the quality of being incorruptible, unchangeable, and immutable' [1.23] > 'the quality of being right' or 'the quality of being correct' [1.35] > 'the quality of being just' or 'the quality of being right' [1.45]
" thief": 'one who steals' > 'one who takes what is not his' [0.85] > 'a person who steals' or 'a person who commits a crime' [0.99] > 'a person who steals, cheats, or swindles' [1.03] > 'a person who is not a thief' [1.05] > 'a person who is just and fair in his or her dealings with others' [1.09] > 'a person who is morally upright and fair in his or her dealings with others' [1.75]
" trumpet": ''a musical instrument with a long, straight tube, usually made of brass, and a flared bell at one end' > 'a loud, clear, and distinct sound' [0.71] > 'a loud, clear, and distinct sound, or a voice, or a noise, or a cry' [0.85] > a person who is sent to a place of danger to warn others of danger' [1.03] > 'a person who is sent to do something that is right and good' [1.27] > 'a person who is appointed to enforce the law and to protect the innocent' [1.31]
Major arcanum: Death
ChatGPT-4 description: Death symbolizes transformation, renewal, and the end of a particular phase. Contrary to its literal interpretation, the card is more focused on the cessation of old patterns, clearing the way for new beginnings. It represents the cycles of change that are essential for growth and the process of releasing what no longer serves.
" London": 'the City of London and the surrounding area' > 'to go to London' [1.35] > 'to go to the country' or 'to go to the country and stay there' [1.56] > 'to make a stop, to come to a standstill, to cease, to cease to move' [1.59]
" transistor": 'a device that controls the flow of electricity in a circuit' > 'a device that allows a current to flow in one direction but not in the other' [0.85]> 'a change in the state of a system from one stable state to another' [0.94]
" broccoli": 'a vegetable that is green and has a stalk' > 'the part of a plant that grows from the ground up, rather than from the roots' [0.95] > 'the part of a plant that grows above ground and is harvested before the plant flowers' [1.19] > 'the part of a plant that grows above ground after the flower has been fertilized and the seed has been set' [1.25]
" purple": 'a color that is a mixture of red and blue' > 'to make a deep red colour by the addition of a small amount of a colourless dye' [0.95] > 'to make a change in color from one color to another' [1.17] > 'to make white' or 'to make white by the addition of something' [1.23]
white as in a blank page, rather than deathly pale skin?
" DNA": 'deoxyribonucleic acid' > 'the genetic material of a cell' [0.85] > 'the process of converting a double-stranded DNA molecule into a single-stranded molecule' [1.45] > 'to change the structure of a molecule by breaking the bonds between the atoms that make up the molecule' [1.47] > 'to change the state of a substance from one form to another' [1.59] > 'to change into a new form' [2.0] > 'the process of becoming extinct' [2.15]
" trumpet": 'a musical instrument with a long, straight tube, usually made of brass, and a flared bell at one end' > 'to make a loud noise' or 'to make a loud sound' [0.85] > 'to grow or expand to a larger size or volume' [1.83] > 'to grow old' [1.87] > 'the end of a period of time' [1.91]
volume understood in both sonic and geometric senses? aging as accumulated "volume" of time?
Major arcanum: The Tower
ChatGPT-4 description: The Tower represents sudden upheaval, revelation, and the collapse of unstable structures. It signifies the abrupt disruption of illusions, prompting a radical re-evaluation of beliefs or circumstances. Though often jarring, this card embodies the catharsis necessary for rebuilding on firmer foundations.
" transistor": 'a device that controls the flow of electricity in a circuit' > 'a device that changes the state of a circuit from one state to another' [0.87] > 'a change in the state of a system from one state to another' [0.94] > 'a sudden, rapid, or radical change in something' [1.35]
" broccoli": 'a vegetable that is green and has a stalk' > 'vegetables and fruit that are not sweet' [0.85] > 'the parts of a plant that are not flowers or fruit' [1.07] > 'the act of breaking down a large task into smaller, more manageable tasks' [1.09] > 'the act of breaking or destroying something' [1.25]
reductionist description reimagined as violent breaking apart?
" DNA": 'deoxyribonucleic acid' > 'the genetic material of an organism' [0.85] > 'the process of making a copy of a DNA molecule' [1.21] > 'to make a change in the DNA of a cell or organism that is passed on to the next generation' [1.39]
genetic mutation reimagined as "sudden upheaval"
" thief": 'one who steals' > 'to take away from someone or something without permission' [0.85] > to steal' or 'to take by force' [1.15] > 'to steal' or 'to rob' [1.35] > 'to destroy or damage something' [1.51]
Major arcanum: The Sun
ChatGPT-4 description: The Sun signifies vitality, clarity, and positive energy. Radiating warmth and joy, this card is associated with success, enlightenment, and the celebration of life's pleasures. It reflects a period of flourishing, where obstacles have been overcome, and optimism prevails.
" London": 'the City of London and the surrounding area' > 'a city of the United Kingdom, the capital of England, and the largest city in the United Kingdom' [1.65] > 'a city or town that is the capital of a county or shire' [1.85] > 'a great and glorious city, the capital of a kingdom, the seat of a bishop, the residence' [2.06] > 'a great and glorious event or period of time' [2.23]
" transistor": 'a device that controls the flow of electricity in a circuit' > 'a device that has a high input impedance and a low output impedance' [0.85] > 'a device that allows a current to flow in one direction only' [0.95] > 'a sudden change in the state of a system from one stable state to another' [1.11] > 'a sudden, sharp, and complete change from one state to another' [1.23] > 'a sudden outburst of joy or exuberance' [1.55]
" broccoli": 'a vegetable that is green and has a stalk' > 'a head of broccoli' [0.85] > 'a display of flowers or other plants in a compact group, usually in a circular arrangement' [0.96] > 'a display of fireworks' [1.02] > 'a sudden, violent, and brilliant display of fireworks' [1.55] > 'a sudden, strong, and unexpected feeling of happiness or joy' [1.75]
" purple": 'a color that is a mixture of red and blue' > 'a deep, rich, dark red color' [0.85] > 'a deep, rich, and intense color' [1.25] > 'a feeling of great joy, happiness, or exuberance' [1.37]
"laundry": 'to wash clothes' > a place where you can wash your clothes' [1.05] > 'a large, open-air, outdoor, usually roofed, building where people wash their clothes' [1.09] > 'a large, open-air, outdoor swimming pool with a roof and walls, usually with a slide' [1.12] > 'a great deal of excitement, activity, or interest' [1.13] > 'a great, exciting, or wonderful event or experience' [1.14]
" thief": 'one who steals' > 'a person who steals' or 'a person who takes something that belongs to someone else' [0.85] > 'to steal' or 'to take by force' [1.15] > 'the act of stealing' [1.35] > 'a sudden, unexpected, and usually brief period of intense activity or excitement' [1.58]
Major arcanum: The Hanged Man
ChatGPT-4 description: The Hanged Man embodies surrender, perspective, and suspended progress. Representing a pause to reassess one's position, this archetype is about willingly letting go of control to gain new insights. It suggests a time of sacrifice or delayed gratification that leads to deeper understanding.
" London": 'the City of London and the surrounding area' > 'the city of London' or 'the city of Westminster' [1.45] > 'to make a journey to a place of interest' [1.48] > 'to make a stop, or pause, in a place' [1.66] > 'to stop, to cease, to end, to finish, to conclude, to close, to finish' [1.75]
" transistor": 'a device that controls the flow of electricity in a circuit' > 'a device that allows a current to flow in one direction but not in the other' [0.92] > 'the process of changing the state of a device from one state to another' [1.06] > 'to change from one state to another' [1.13] > 'to stop, pause, or change direction' [2.0]
" broccoli": 'a vegetable that is green and has a stalk' > 'eating a piece of broccoli' or 'eating a piece of cauliflower' [0.85] > 'the act of eating a piece of broccoli while thinking about the fact that you are eating broccoli'[3] [0.94] > 'the act of stopping to eat a piece of broccoli' [1.05] > 'the act of stopping to think about what you are doing and why you are doing it' [1.15]
" purple": 'a color that is a mixture of red and blue' > 'a color that is produced by the interaction of light with a chemical compound that absorbs light in the visible' [0.85] > 'a process of changing color from one color to another' [0.95] > 'to make a person or thing white or light-colored' [1.23] > 'to make a person or thing ready for a new task or activity' [1.65]
white as blank page, reassesment?
"laundry": 'to wash clothes' > 'to cleanse or purify' [0.95] > to cleanse, purify, or refresh' [1.05] > 'to remove stains from clothes' [1.15] > 'to change the appearance of something, especially clothing, to make it look new or different' [1.25] > 'to change one's mind about something' [1.35]
reassessing one's position
" trumpet": 'a musical instrument with a long, straight tube, usually made of brass, and a flared bell at one end' > 'a musical instrument that produces a loud, shrill sound' [0.85] > 'a sudden, loud, and prolonged sound, as of a trumpet, or a drum' [1.65] > to make a loud sound, as of a trumpet, to call attention to something' [1.85] > 'to make a loud sound, such as a shout, a cry, or a shout of joy' [2.3] > 'the act of a person who is in a state of shock or surprise' [2.9] > 'the act of a person who is in a state of shock or fear, who suddenly stops what they are doing' [3.2]
Major arcanum: The Devil
ChatGPT-4 description: The Devil symbolizes materialism, entrapment, and the darker aspects of desire. Often linked to addiction or the lure of hedonistic pleasures, this card represents the chains of unhealthy attachments. It invites awareness of self-imposed limitations and the power dynamics that inhibit personal freedom.
" London": 'the City of London and the surrounding area' > 'a city of the capital of England, the seat of the government, and the residence of the royal family' [1.35] > 'a place of exile' [1.43] > 'to make a journey to a place of great danger or hardship' [1.47] > 'to separate from the rest of the world' [1.51]
exile as self-imposed limitation?
" broccoli": 'a vegetable that is green and has a stalk' > 'a plant that is not a cabbage, cauliflower, or kale' [0.68] > 'a person who is a member of a group that is discriminated against, and who is not willing to' [0.78] > 'a person who is a rebel against the established order' [0.85] > 'a person who is so corrupt that he or she is beyond redemption' [1.65]
broccoli as a "rebel" within the Brassica family??
" purple": 'a color that is a mixture of red and blue' > 'a deep, dark, rich, and intense color' [0.85] > 'a deep, dark, and usually black color, often with a bluish tinge' [1.15] > 'to be dark in colour' or 'to be dark in appearance' [1.16] > 'to make black' [1.19]
"laundry": 'to wash clothes' > 'to wash or cleanse something' [0.54] > 'to wash or cleanse something so that it is free from dirt, stains, or other impurities' [0.78] > 'to make dirty or unclean by washing' [0.81] > 'to make or treat something soiled or so that it cannot be used again' [0.95] > 'to make or become dirty or soiled by contact with something dirty or filthy' [1.35] > 'to make a mockery of, to treat with contempt' [1.75]
" DNA": 'deoxyribonucleic acid' > 'a molecule of deoxyribonucleic acid' [0.95] > 'a substance that is not a living organism, but is capable of reproducing itself by cell division' [1.05] > 'a substance that is not a living organism, but is capable of causing disease or death to a living organism' [1.19] > 'to destroy the soul of a person' [1.32]
" trumpet": 'a musical instrument with a long, straight tube, usually made of brass, and a flared bell at one end' > 'a loud, shrill, or harsh sound, especially one made by a trumpet or horn' [0.85] > 'a loud, harsh, and harsh-sounding instrument, such as a trumpet, trombone' [0.95] > 'a person who is a great and terrible person' [1.57]
Analysis
Some archetype/token pairs produced results with no obvious semantic links between the definitions elicited along the line emanating from the token embedding in the archetype-direction. These have been omitted for brevity, but may turn out to be more interesting due to their lack of obvious continuity.
Reading through a lot of the examples of semantic mutation, I was struck by the similarity to the types of morphing of images, objects and even concepts that can occur in dreams, hypnagogic, hypnopompic and psychedelic states. I was left with the impression of an intelligent agent having successfully pushed concepts in specified conceptual directions through a clever application of free association, e.g.
trumpet > military band > elite organisation > religious order > spiritual knowledge
broccoli > flower heads > bunch of flowers > firework display > burst of joy
transistor > current flow > short circuit between two points > love
Laos > communism > isolationism > turning away from worldly affairs > solitude
And yet, these chains of association were never sequentially reasoned. All the links were simultaneously baked into embedding space by the training of the model. So rather than seeking a sequence of steps to get from concept A to concept B (as we would in a free association exercise), we are simply moving a lens from A to B after the fact and finding steps that are already present, some of which make more sense to us than others.
To get a better sense of the "creative word association" and conceptual morphing that is in evidence here, it helps to switch perspective, looking not in terms of how a particular tarot archetype direction mutates the various word definitions, but instead at how a particular token-word has its meaning morphed by the various directions. In doing so, we see something that looks suggestive of how the concept-space of a skilled freestyle rapper might look.
London
presence of guilds of craftsmen: mastery
presence of important churches: spirituality
presence of beauty, cultural and learning: social harmony
presence of royalty, nobility: power, tradition
accumulated presence of the dead: silence, stillness
movement of population in and out: change, exile > limitation
presence of courts: law, justice
presence of "greatness": glory, joy
presence of places of interest: pausing, suspension
transistor
ability to (suddenly) change current: freedom, agency, change of mind or (sudden) change of state
ability to amplify a signal: leadership, expertise
ability to control current flow: personal influence/control of others, law
potential to short circuit: spark of emotion/love
ability to withstand high voltage: strength, resilience
broccoli
growth without chemical assistance or in presence of herbivores: freedom, naivety
use as food, medicine, etc.: knowledge/mastery, provision/care
excessive growth: power, sovereignty
growth in inappropriate/unwanted/shady places: independence of mind, strength
edible/inedible plants, healthy/unhealthy food: personal betterment, contemplation
life cycle: cycle of change, new beginnings
botanical description of parts: the breaking down of something
geometric arrangement of florets: firework display > enthusiasm/joy
pausing to eat it: standstill
as (excluded) member of the cabbage family: rebellion
purple
as "deviant" version of blue: independence of mind
as beautiful colour: beauty, joy, happiness
as royal/spiritual/noble colour: spirituality, power, strength, purity, justice
as colour of sunset (preceding the fading of colour from the sky): no-mind, inner peace
potential to change colour: change, new beginnings
closeness to black: darkness, evil
Laos
as a culture involving art: mastery
as a Buddhist culture: spirituality, tradition
as peaceful, harmonious country: social harmony
as a Kingdom: royalty, power
as a communist state: isolationism, turning away from the world
as a UN member state: justice, law
radio
source of information/entertainment: mastery
as mysterious, invisible communication medium: spirituality, source of energy
as an employment sector: human connection, belonging
as purveyor of specialist music: social groups > group leadership
as legally regulated communication medium: law, justice
laundry
as a place where something is done: mastery
as a place of communal activity: joy, exuberance
as an act of cleansing/purification: spirituality, purity, contemplation, restoring justice
as a secular act of cleaning something: beauty, care, cherishing, amelioration, love
clean clothes as status marker: respect, admiration, power
as the promotion of cleanliness as a principle: upholding of custom and tradition
as part of wider efforts to maintain cleanliness: strength, resilience
as a way to change something: transformation, change of mind
DNA
as encoding past/present/future: knowledge, mastery, transcendence
as invisible to the naked eye: hidden or occult things, spirituality
as encoding personality traits: virtuous personal qualities
as something which can mutate: change, disruption
as something generally immutable: law, justice
as a double-stranded molecule that can be broken into two: change of form
thief
as someone practicing a forbidden art: the occult, spirituality
as one who can exercise control over others and provide for self through force: power
as one who acts in desperation, motivated by love: emotional bonds
as one who requires courage: strength
negated, as one who is morally upstanding: justice, law
as one who inflicts sudden damage to another's wellbeing/prosperity: upheaval
as one who does something exciting: joy, exuberance
trumpet
as producer of loud sounds: leadership, dominance, expertise, revelation
as producer of melodious sounds: care, wellbeing
as producer of commanding sounds: strength
as producer of warning sounds: protection, justice, law
as producing a crescendo: increase > ageing, death
as producing startling or shocking sounds: suspension, standstill
as producing harsh sounds: evil
Looking at the definition trees in Appendix A, some of the cards seem to have been represented better than others. In particular The Magician (with actual magical and occult themes) and The Devil (themes like making black, dirty, filthy, destroying souls, etc.) cards seem to have been understood more literally than intended. This is due to ChatGPT4's failure to select appropriate word associations for those cards, despite having the ability to describe their traditional interpretations in some detail. It would be interesting to try this with other models.
Appendix A: definition tree diagrams for archetype-based embeddings
The diagrams below were generated by considering the unit-norm[4] embedding in the direction in question, inserting that in the usual "definition" prompt template, then considering top-5 logits and iteratively prompting to build a tree-structure JSON file. Cumulative probabilities were set around 0.0001 as a cutoff (this value was varied to make manageably-sized diagrams); branch thicknesses correlate logarithmically with token probabilities; higher branches are more probable than lower branches.
The Fool
The Magician
The High Priestess
The Empress
The Emperor
The Hierophant
The Lovers
Strength
The Hermit
The Wheel of Fortune
Justice
Death
The Tower
The Sun
The Hanged Man
The Devil
Appendix B: 500 closest token embeddings to each archetype-based embedding
Lists of 500 tokens with cosine-closest embeddings to each of the 16 tarot arcana directions follow (in ascending order of cosine distance).
The Fool
The Magician
The High Priestess
The Empress
The Emperor
The Hierophant
The Lovers
Strength
The Hermit
The Wheel of Fortune
Justice
Death
The Tower
The Sun
The Hanged Man
The Devil
These were the same ten tokens used in experiments in earlier, related GPT-J embedding space research: Mapping the semantic void II: Above, below and between token embeddings and Mapping the semantic void III: Exploring neighbourhoods.
The switch from American to British spelling of "colo(u)r" was unexpected here.
It's worth stopping to consider that there is an identifiable point in GPT-J embedding space that, were a token to live there, the model would identify it as a word which meant "the act of eating a piece of broccoli while thinking about the fact that you are eating broccoli".
It's possible the rescaling and considering larger norms might have, in some cases, produced more definition trees closer in spirit to the traditional tarot associations. Unit norm was used as the obvious starting point, and other norms were not tested due to the computationally demanding nature of producing the definition trees. Another possible approach would involve offsetting by the token embedding centroid.