This website requires javascript to properly function. Consider activating javascript to get access to all site functionality.
LESSWRONG
Tags
LW
Login
Subscribe
Discussion
(0)
Glitch Tokens
Subscribe
Discussion
(0)
Written by
CronoDAS
,
Raemon
last updated
18th Apr 2023
Glitch Tokens are tokens in a language model that cause anomalous output, such as
SolidGoldMagikarp.
Posts tagged
Glitch Tokens
Most Relevant
10
680
SolidGoldMagikarp (plus, prompt generation)
Ω
Jessica Rumbelow
,
mwatkins
2y
Ω
206
10
192
The ‘ petertodd’ phenomenon
mwatkins
2y
50
6
91
SolidGoldMagikarp III: Glitch token archaeology
Ω
mwatkins
,
Jessica Rumbelow
2y
Ω
35
3
139
Anomalous tokens reveal the original identities of Instruct models
Ω
janus
,
jdp
2y
Ω
16
3
113
SolidGoldMagikarp II: technical details and more recent findings
Ω
mwatkins
,
Jessica Rumbelow
2y
Ω
45
3
109
' petertodd'’s last stand: The final days of open GPT-3 research
mwatkins
1y
16
2
114
Mapping the semantic void: Strange goings-on in GPT embedding spaces
mwatkins
1y
31
2
85
The "spelling miracle": GPT-3 spelling abilities and glitch tokens revisited
mwatkins
2y
29
2
71
SmartyHeaderCode: anomalous tokens for GPT3.5 and GPT-4
Ω
AdamYedidia
2y
Ω
18
2
40
What's up with all the non-Mormons? Weirdly specific universalities across LLMs
mwatkins
10mo
13
2
38
Glitch Token Catalog - (Almost) a Full Clear
Lao Mein
4mo
3
2
37
A New Class of Glitch Tokens - BPE Subtoken Artifacts (BSA)
Lao Mein
4mo
7
2
34
Linear encoding of character-level information in GPT-J token embeddings
mwatkins
,
Joseph Bloom
1y
4
2
21
Nokens: A potential method of investigating glitch tokens
Hoagy
2y
0
2
12
Exploring the petertodd / Leilan duality in GPT-2 and GPT-J
mwatkins
1mo
1