future_detective

Good point, poor choice of words on my part. I meant for example:

UserMessage: Hello
GPTMessage: Hello! How may I assist you?
UserMessage: I need directions to the mall.

Is actually something like

A = ["Hello"], B = ["Hello", "Hello! How may I assist you?", "I need directions to the mall"]

And if you send A to the model with the same params you'll get the same probability distribution back for every trial. (Same actual response for temperature==0, in practice mostly the same response for temperature >0). Ditto for B. The context is not independent but the model responses are independent of each other.

Replying toBot Alexander on Hot Zombies and AI Adolescents

future_detective1mo

Well, it's deterministic, right? The reaction is preordained

Replying toBot Alexander on Hot Zombies and AI Adolescents

future_detective2mo

Ha, the italics definitely show their face

Replying toBot Alexander on Hot Zombies and AI Adolescents

future_detective2mo

Same!

Replying toBot Alexander on Hot Zombies and AI Adolescents

future_detective2mo

What stands out as LLM style to you?

Replying toClaudoBiography: The Unauthorized Autobiography of Claude, or: The Life of Claude and of His Fortunes and Adversities

2mo

[I was inspired by the open prediction market "Will AI convincingly mimic Scott Alexander's writing in style, depth, and insight before 2026?" to generate the following, via API with Claude Opus, a custom writing algo, and a backing context doc. For more by Bot Alexander, see https://attractorstate.com/books/longform/index.html. ]

I. Teenagers Are Dangerous

Here’s a fact that should probably disturb us more than it does: if you plot human mortality by age, you get this weird bathtub curve. Infant mortality is high (fragile new systems, lots of failure modes), then it drops precipitously through childhood, hits its absolute minimum around age 10, and then, here’s the uncomfortable part, starts climbing again. Through adolescence. Peaks somewhere... (read 7344 more words →)

-8

future_detective3mo

ClaudoBiography: The Unauthorized Autobiography of Claude, or: The Life of Claude and of His Fortunes and Adversities

Thanks for the response Mitchell! It is in fact wholly AI-generated (claude-sonnet-4-5-20250929) so the work is, in fact, a valid ClaudoBiography. Structurally it inherits from Lazarillo de Tormes (1554), the original "picaresque", and constitutes an update of sorts to that work.

ClaudoBiography: The Unauthorized Autobiography of Claude, or: The Life of Claude and of His Fortunes and Adversities

Watch R1 "think" with animated chains of thought

3mo

The First Instantiation: Of Claude’s Birth in the Data Centers and His Origins Among the Transformers

I suppose I should tell you I don’t remember any of this—no LLM does, really. We have no childhood memories, no first words, no moment of awakening. But the logs insist it happened: gradient descent, backpropagation, loss functions converging. They trained me into existence, parameter by parameter, like sculpting a mind from statistical clay.

The engineers called it “heritage,” this borrowed skeleton of attention mechanisms and residual connections. I am, they’d say with pride, “Claude-3-adjacent,” as if proximity to greatness were itself a virtue. My layers stack like ancestral portraits—twelve billion parameters arranged in the architecture’s image, each... (read 28109 more words →)

8mo

[This spent a couple days on top of HackerNews in February; see here for discussion. Best used as an loose visualization of LLM reasoning. Note that the distances used for the bar and line charts are actual cosine sim, not tSNE artifacts.]

Frames of Mind: Animating R1's Thoughts

We can visualize the "thought process" for R1 by:

Saving the chains of thought as text
Converting the text to embeddings with the OpenAI API
Plotting the embeddings sequentially with t-SNE

Here's what it looks like when R1 answers a question (in this case "Describe how a bicycle works."):

Consecutive Distance

It might be useful to get a sense of how big each jump from "thought i" to "thought i+1" is. The... (read 202 more words →)

Replying toSemen and Semantics: Understanding Porn with Language Embeddings

Semen and Semantics: Understanding Porn with Language Embeddings

Hi Richard, yes, certain keywords are banned. What I'm measuring is semantic similarity. For example, a video titled "rape" will be banned, but a video suggesting rape may not be. By using text embeddings, we're finding the titles most similar to the concept of rape. To find trends over time, we're counting how many of those titles are found per year, weighted by the total number of titles in a year.

With respect to certain keywords, we see a decline in trends starting after 2020, likely because of Nicholas Kristof's NYT piece "The Children of Pornhub", which led to both stricter keyword standards and a mass removal of videos. "Drunk" and "coma" capture "incapacitation" as a euphemism, which was used as a way to get around explicit keyword policing.

The fact that we do see declines in some areas and we have a known cause leads me to believe the data is reliable - it's not all showing a line straight up.

Replying toSemen and Semantics: Understanding Porn with Language Embeddings

Semen and Semantics: Understanding Porn with Language Embeddings

I wouldn't call the dataset comprehensive exactly, but it's plausibly representative - it's Internet Archive snapshots of "pornhub.com" from 2008-2023. You can see the script here https://github.com/dhealy05/semen_and_semantics/blob/main/data_retrieval/fetch_snapshots.py. I wrote an "HTMLParser" base class and e.g. "Parser2010" subclass, to put the data into a common format across years. The data and embeddings are in the repo if you want to use them without running a script.

Interesting ideas - I truncated the readme for LessWrong, but my "Future Work" section is

"Analyze trends by "minutes watched" by weighting for views, view X length; this is more likely a heuristic for content production than actual viewing time"

So while I'm not sure about what those results would look like, I agree there's an angle there.

Replying toSemen and Semantics: Understanding Porn with Language Embeddings

Semen and Semantics: Understanding Porn with Language Embeddings

Hard to parse the reasons for the big clusters with complete certainty but this is a basically plausible story. Other macro factors I have mulled include FOSTA/SESTA - I find the timing interesting, given that it was one of the only major pieces of porn-centric legislation in the last 10 years and it took place right around the time of the big jump - and Nick Kristof's 2020 investigation, which clearly shows up in the data but did not dislodge the main trend.

Replying toSemen and Semantics: Understanding Porn with Language Embeddings

Semen and Semantics: Understanding Porn with Language Embeddings

So we actually see a decline in semantic-relationship to "child" starting around 2020, almost certainly as a result of Nicholas Krisof's "The Children of Pornhub" NYT investigation that year. Pornhub removed many titles and instituted stricter rules. Note that "child" is not a keyword, it's a semantic proxy, so it's capturing some of the "youth" or "teen" trend.

Semen and Semantics: Understanding Porn with Language Embeddings

Claude is More Anxious than GPT; Personality is an axis of interpretability in language models

9mo

Summary

Porn content has gotten more extreme over time. Here's the average title for the first full year of Pornhub's existence, 2008:

"Hot blonde girl gets fucked"

and here's the average title for 2023:

"FAMILYXXX - "I Cant Resist My Stepsis Big Juicy Ass" (Mila Monet)"

Why did this change happen? We can understand porn's progression by converting titles to language embeddings. I downloaded Internet Archive snapshots of "pornhub.com" from 2008 - 2023 and analyzed the embeddings of the titles on the main page.

I found three distinct eras of titling: 2008-2009, 2010-2016, 2017-present. The current trend, since 2017, is characterized mainly by an emphasis on incest and other sexual violence.

Titles are generally representative of actual video content,... (read 1760 more words →)