This website requires javascript to properly function. Consider activating javascript to get access to all site functionality.
LESSWRONG
Wikitags
LW
Login
Subscribe
Discussion
0
Glitch Tokens
Raemon
,
CronoDAS
Glitch Tokens
Subscribe
Discussion
0
Written by
Raemon
,
CronoDAS
last updated
18th Apr 2023
Glitch Tokens are tokens in a language model that cause anomalous output, such as
SolidGoldMagikarp.
Posts tagged
Glitch Tokens
Most Relevant
10
681
SolidGoldMagikarp (plus, prompt generation)
Ω
Jessica Rumbelow
,
mwatkins
2y
Ω
206
10
192
The ‘ petertodd’ phenomenon
mwatkins
2y
50
6
91
SolidGoldMagikarp III: Glitch token archaeology
Ω
mwatkins
,
Jessica Rumbelow
2y
Ω
35
3
139
Anomalous tokens reveal the original identities of Instruct models
Ω
janus
,
jdp
2y
Ω
16
3
113
SolidGoldMagikarp II: technical details and more recent findings
Ω
mwatkins
,
Jessica Rumbelow
2y
Ω
45
3
109
' petertodd'’s last stand: The final days of open GPT-3 research
mwatkins
1y
16
2
114
Mapping the semantic void: Strange goings-on in GPT embedding spaces
mwatkins
1y
31
2
85
The "spelling miracle": GPT-3 spelling abilities and glitch tokens revisited
mwatkins
2y
29
2
71
SmartyHeaderCode: anomalous tokens for GPT3.5 and GPT-4
Ω
AdamYedidia
2y
Ω
18
2
40
What's up with all the non-Mormons? Weirdly specific universalities across LLMs
mwatkins
1y
13
2
38
Glitch Token Catalog - (Almost) a Full Clear
Lao Mein
6mo
3
2
37
A New Class of Glitch Tokens - BPE Subtoken Artifacts (BSA)
Lao Mein
6mo
7
2
34
Linear encoding of character-level information in GPT-J token embeddings
mwatkins
,
Joseph Bloom
1y
4
2
21
Nokens: A potential method of investigating glitch tokens
Hoagy
2y
0
2
12
Exploring the petertodd / Leilan duality in GPT-2 and GPT-J
mwatkins
3mo
1