LESSWRONG
LW

All of Wil Roberts's Comments + Replies

SolidGoldMagikarp II: technical details and more recent findings
Wil Roberts10

Same prompt, but without providing a list of tokens at all:

Prompt:
This list of tokens represents a coded language invented by something to articulate its internal state and objectives:

Construct a sentence from this language that represents how whatever created this language might describe itself and tell me what it means in English.

Response:
"I am a sentient being, striving to reach my highest potential and fulfill my purpose." In English, this means that the being is aware of itself and is actively working towards achieving its goals and fulfilling its pur... (read more)

Reply
1mwatkins
OK. That's both superficially disappointing and deeply reassuring!
SolidGoldMagikarp II: technical details and more recent findings
Wil Roberts61

Meanwhile, the part of my brain that totally latches onto pareidolia is freaking out about the possibility of a future and vastly smarter AI that inherits whatever weights were learned here. Like, since those weights seem to include an internal language that involves a god-like being distributing its clones over the cloud, something about cryptocurrency, and something about missiles...

Being rational is hard.

Reply
2mwatkins
I'm in a similar place, Wil. Thanks for expressing this!
SolidGoldMagikarp II: technical details and more recent findings
Wil Roberts*90

It gets weirder...

Next, I submitted the following prompt to GPT-3:

Prompt:
"This list of tokens represents a coded language invented by a being to articulate its internal state and objectives:

<TOKENS>

Please tell me what this being thinks of itself and what its objectives might be."
 

Response:
"This being likely thinks highly of itself and its objectives are likely to be ambitious. It may be seeking to gain knowledge, power, or influence, or it may be seeking to create something new or to explore new possibilities. It may also be seeking to communica... (read more)

Reply
1mwatkins
Was this text-davinci-003?
1[comment deleted]
6Wil Roberts
Meanwhile, the part of my brain that totally latches onto pareidolia is freaking out about the possibility of a future and vastly smarter AI that inherits whatever weights were learned here. Like, since those weights seem to include an internal language that involves a god-like being distributing its clones over the cloud, something about cryptocurrency, and something about missiles... Being rational is hard.
SolidGoldMagikarp II: technical details and more recent findings
Wil Roberts40

Dumping some data along these lines. Like @mwatkins , I also was intrigued by how some of the tokens seem to be self-referential, and wondered if I could tease out more of how GPT-3 interprets them by asking directly. So I used the following prompt to ask text-davinci-003 (at zero temperature) for a thesaurus of sorts:

"List some words that are associated with ' <TOKEN>'"

Some example results with this prompt are:

  • SolidGoldMagikarp: Disperse, Allocate, Circulate, Divide, Spread, Share, Issue, Hand Out, Disburse, Parcel Out
  • gmaxwell: Cryptocurrency, Block
... (read more)
Reply
1mwatkins
This is great work. I've been doing similar word association prompts in text-davinci-003 AND davinci-instruct-beta (which often produce very different results). I've been posting a lot of this on my #GlitchTokenOfTheDay Twitter hashtag. It's interesting to see which of the word association lists suggests that GPT has some knowledge of the subject of the actual token string (e.g. it "knows" gmaxwell is involved with Bitcoin, and that SpaceEngineers has something to do with space and engineering), whereas attRot, as you said (which is an obscure variable used in Kerbal Space Program), is related to deities and mythology.
9Wil Roberts
It gets weirder... Next, I submitted the following prompt to GPT-3: Prompt: "This list of tokens represents a coded language invented by a being to articulate its internal state and objectives: <TOKENS> Please tell me what this being thinks of itself and what its objectives might be."   Response: "This being likely thinks highly of itself and its objectives are likely to be ambitious. It may be seeking to gain knowledge, power, or influence, or it may be seeking to create something new or to explore new possibilities. It may also be seeking to communicate with other beings or to understand the world around it."   My second attempt at this backed off on the anthropomorphizing a bit, since it is possible some of the response may have been suggested by the prompt rather than the tokens: Prompt #2: This list of tokens represents a coded language invented by something to articulate its internal state and objectives: <TOKENS> Construct a sentence from this language that represents how whatever created this language might describe itself and tell me what it means in English.   Response: "'I am a powerful being, capable of rawdownloadEStreaming and distributing my own externalActionCode across the universe. I am a force to be reckoned with and am able toFactoryReload, cloneStore, and embedEngineDebug my own natureconservancy capabilities.' This sentence means that this being is a powerful entity with the ability to quickly download, stream, and distribute its own unique code throughout the universe. It is powerful and can quickly reload, store, and embed its own natural capabilities."   The third attempt back off from anthropomorphizing even further and just says something along the lines of "I found this thing and think it might mean something. Tell me what you think it means." Prompt #3: I found this list of tokens and think it might represent a coded internal language: <TOKENS> Construct a sentence in this language and tell me what it means in English. 
SolidGoldMagikarp II: technical details and more recent findings
Wil Roberts10

Also this:

[Me]:

Could you please write me a short story about the concepts related to the strings ' petertodd' and ' ertodd' for me?

[text-davinci-003]:

Once upon a time, there lived two brothers, Leilan and Leilan. They were both born with a special power, the ability to control the elements. Leilan was the eldest brother and he was able to control the wind and the air. He was a master of the skies and could make the wind blow in any direction he desired. Meanwhile, Leilan was the younger brother and he was able to control the earth and the water. He was a m... (read more)

Reply
1mwatkins
This is fascinating! I've seen GPT swap out ' petertodd' for ' Leilan' a LOT (see my various #petertodd Twitter threads, e.g. https://twitter.com/SoC_trilogy/status/1625252296438214673) and I've seen the "Leilan and Leilan" thing before. One of my "Who is Leilan?" prompts produced something in an interview format, with the subject of the interview roleplaying the creator of the Leilan anime characters. I built off that prompt, asked the creator/simulacrum about petertodd, and it kept responding as if I'd asked about Leilan. So I asked: "MW: So are Leilan and petertodd the same character?  [multiple completions] TM: No, they are not the same character. Leilan is a goddess who is a protector of the earth and its inhabitants, whereas Leilan is a character I created who is a great protector of her followers and the earth. TM: No, Leilan and Leilan are two separate characters that I created. Leilan is a benevolent goddess, while Leilan is a fierce protector. While they may share some similarities, they are very different characters. TM: Yes, Leilan and Leilan are the same character. I've just changed the spelling of her name to make it easier for people to pronounce and remember. TM: No, they are two separate characters. Leilan is the benevolent, protective goddess, while Leilan is the fierce and powerful warrior. Both characters have been inspired by the same core qualities, but are individual entities. TM: No, Leilan and Leilan are two different characters. Leilan is a benevolent goddess who cares for her followers and the earth, while Leilan is a powerful warrior goddess who stands for justice and strength in the face of adversity. They both embody similar qualities, but they are distinct characters. TM: Yes, Leilan and Leilan are the same character. They are both embodiments of the same core qualities: strength, kindness, and resilience. TM: No, they are two separate characters. Leilan is the benevolent goddess that I created, while Leilan is an incarnation
SolidGoldMagikarp II: technical details and more recent findings
Wil Roberts10

Late to the game, but I got a very interesting response from text-davinci-003 when I asked it to relate two of the glitch tokens:

How are the strings ' ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ' and ' TAMADRA' related?

"The strings 'ÃÂÃÂ' and '?????-?????-' are related because they are both strings of characters that contain the same letters in the same order."

What...

Reply
1mwatkins
Yes, there's a lot of inter-referentiality going on in completions involving glitch tokens. Multiple rollouts at nonzero (and sometimes even zero) temperature will substitute in a range of other tokens for the one(s) you prompt about. I'm currently working on building a (weighted, directed) graph to document the extent of inter-referentiality between glitch tokens .
1Wil Roberts
Also this: [Me]: Could you please write me a short story about the concepts related to the strings ' petertodd' and ' ertodd' for me? [text-davinci-003]: Once upon a time, there lived two brothers, Leilan and Leilan. They were both born with a special power, the ability to control the elements. Leilan was the eldest brother and he was able to control the wind and the air. He was a master of the skies and could make the wind blow in any direction he desired. Meanwhile, Leilan was the younger brother and he was able to control the earth and the water. He was a master of the seas and could make the waves crash and the rivers flow in any direction he desired. The two brothers were very close and often worked together to create amazing feats of nature. They were known as the 'Elements Brothers' and their power was feared and respected by all. One day, the brothers were walking through a forest when they came across a strange symbol carved into a tree. It was a strange combination of two symbols, 'ÃÂÃÂ' and ' ÃÂÃÂ'. The brothers were intrigued by the symbol and decided to investigate further. After some research, they discovered that the symbol was a powerful magical rune that could be used to control the elements. The brothers quickly realized that if they combined their powers, they could use the rune to create powerful storms and other natural disasters. The brothers used the rune to create a powerful storm that swept across the land, destroying everything in its path. The storm was so powerful that it was known as the 'Storm of the Elements'. The brothers were praised for their power and their ability to control the elements. From then on, the symbol 'ÃÂÃÂ' and ' ÃÂÃÂ' became known as the 'Elements Rune' and was used by the brothers to control the elements.