Top postsTop post
charlieoneill
244
6
6
For people who care about falsifiable stakes rather than vibes TL;DR All timeline arguments ultimately turn on five quantitative pivots. Pick optimistic answers to three of them and your median forecast collapses into the 2026–2029 range; pick pessimistic answers to any two and you drift past 2040. The pivots (I...
The idea that wealth should be inherited is so ingrained in our thinking that we rarely question it. But step back for a moment, and it becomes a curious thing—this notion that assets, land, fortunes should be passed down from one generation to the next without scrutiny. The debate over...
> Chris Olah's “What is a Linear Representation? What is a Multidimensional Feature?” (July Circuits Update) prompted a moment of pause for me regarding the term "one-dimensional feature." I initially conflated that phrase with the number of dimensions in the activation space (for example, the 768 dimensions in GPT‑2 Small)....
I've noticed something about how humans and language models work together. There's a pattern that emerges whenever we collaborate effectively. It goes like this: Someone has an initial idea (step 1). An LLM can then generate variations and connections around that idea (step 2). A human needs to look at...
> "What is the use of having developed a science well enough to make predictions if, in the end, all we're willing to do is stand around and wait for them to come true?" F. SHERWOOD HOWLAND in his speech accepting the Nobel Prize in Chemistry in 1995. > "Once...
Executive Summary * I try vector-quantised autoencoders (VQ-VAEs) as an alternative compression scheme of transformer activations (as opposed to something like a sparse autoencoder). * Whilst people have danced around this idea before, discrete quantisation has only ever been tried in the actual transformer architecture itself, rather than on cached...