This website requires javascript to properly function. Consider activating javascript to get access to all site functionality.
LESSWRONG
is fundraising!
Tags
LW
$
Login
Glitch Tokens
Edit
History
Subscribe
Discussion
(0)
Help improve this page
Edit
History
Subscribe
Discussion
(0)
Help improve this page
Glitch Tokens
Random Tag
Contributors
6
CronoDAS
6
Raemon
Glitch Tokens are tokens in a language model that cause anomalous output, such as
SolidGoldMagikarp.
Posts tagged
Glitch Tokens
Most Relevant
10
677
SolidGoldMagikarp (plus, prompt generation)
Ω
Jessica Rumbelow
,
mwatkins
2y
Ω
206
Review
10
192
The ‘ petertodd’ phenomenon
mwatkins
2y
49
Review
6
91
SolidGoldMagikarp III: Glitch token archaeology
Ω
mwatkins
,
Jessica Rumbelow
2y
Ω
32
Review
3
139
Anomalous tokens reveal the original identities of Instruct models
Ω
janus
,
jdp
2y
Ω
16
Review
3
111
SolidGoldMagikarp II: technical details and more recent findings
Ω
mwatkins
,
Jessica Rumbelow
2y
Ω
45
3
109
' petertodd'’s last stand: The final days of open GPT-3 research
mwatkins
1y
16
2
114
Mapping the semantic void: Strange goings-on in GPT embedding spaces
mwatkins
1y
31
2
85
The "spelling miracle": GPT-3 spelling abilities and glitch tokens revisited
mwatkins
1y
29
2
71
SmartyHeaderCode: anomalous tokens for GPT3.5 and GPT-4
Ω
AdamYedidia
2y
Ω
18
2
40
What's up with all the non-Mormons? Weirdly specific universalities across LLMs
mwatkins
8mo
13
2
38
Glitch Token Catalog - (Almost) a Full Clear
Lao Mein
3mo
3
2
37
A New Class of Glitch Tokens - BPE Subtoken Artifacts (BSA)
Lao Mein
3mo
7
2
34
Linear encoding of character-level information in GPT-J token embeddings
mwatkins
,
Joseph Bloom
1y
4
2
21
Nokens: A potential method of investigating glitch tokens
Hoagy
2y
0
2
10
Exploring the petertodd / Leilan duality in GPT-2 and GPT-J
mwatkins
7d
0