LESSWRONG
is fundraising!
Tags
LW
$

Glitch Tokens

•

Applied to Exploring the petertodd / Leilan duality in GPT-2 and GPT-J by mwatkins 3d ago

•

Applied to Glitch Token Catalog - (Almost) a Full Clear by Lao Mein 3mo ago

•

Applied to A New Class of Glitch Tokens - BPE Subtoken Artifacts (BSA) by Lao Mein 3mo ago

•

Applied to LLMs Universally Learn a Feature Representing Token Frequency / Rarity by Sean Osier 6mo ago

•

Applied to What's up with all the non-Mormons? Weirdly specific universalities across LLMs by mwatkins 8mo ago

•

Applied to An examination of GPT-2's boring yet effective glitch by MiguelDev 8mo ago

•

Applied to Anomalous tokens reveal the original identities of Instruct models by MiguelDev 1y ago

•

Applied to ' petertodd'’s last stand: The final days of open GPT-3 research by mwatkins 1y ago

•

Applied to Mapping the semantic void: Strange goings-on in GPT embedding spaces by jacobjacob 1y ago

•

Applied to Linear encoding of character-level information in GPT-J token embeddings by mwatkins 1y ago

•

Applied to <|endoftext|> is a vanishing text? by MiguelDev 1y ago

•

Applied to The "spelling miracle": GPT-3 spelling abilities and glitch tokens revisited by duck_master 1y ago

•

Applied to (redacted) Anomalous tokens might disproportionately affect complex language tasks by Nikola Jurkovic 2y ago

•

Applied to A Search for More ChatGPT / GPT-3.5 / GPT-4 "Unspeakable" Glitch Tokens by Martin Fell 2y ago

CronoDAS v1.1.0Apr 18th 2023 GMT (+18/-25) 6

Glitch Tokens are tokens in a language model that cause anomalous output, such as ~~SolidGoldMagickCarp(sp?)~~ SolidGoldMagikarp.

•

Applied to SmartyHeaderCode: anomalous tokens for GPT3.5 and GPT-4 by AdamYedidia 2y ago

•

Applied to The ‘ petertodd’ phenomenon by mwatkins 2y ago

•

Applied to SolidGoldMagikarp III: Glitch token archaeology by plex 2y ago