Brendan Long

Exercise is hard but it's even harder if you have to use your brain and muscles at the same time. I wish a personal trainer would just teleport into my house whenever I work out, tell me exactly what to do, and then record my progress (and complaints) to improve...

Instead of using static position increments (+1) per token, RoPE-based language models can learn per-token and per-layer position increments. This has minimal effect on model performance but allows us to see what the model thinks the distance is between each position and how this varies per-layer. Example sentence with each...

I'm working on an experiment comparing the internal representations of two architectures when solving a sequential algorithm, but training models to use a sequential algorithm is surprisingly hard. The optimization landscape makes it easier for models to learn parallel algorithms or memorize lookup tables, so I needed to make some...

I've been working on two fairly large vibe-coded apps, and my process has converged on: 1. Write a GitHub issue 2. (If complicated enough) tell an agent to make a plan and then update the issue 3. Have another agent read the issue and implement it As the features get...

LessWrong's RSS feed includes all recently published articles by default, but it has a bunch of undocumented features available with query params. Warning: The only officially supported options are view=frontpage, view=curated, and karmaThreshold. Other options may break in the future. * You can request only posts above a certain karma...

I posted a few months ago about vibe-coding an RSS reader. The mood on the internet seems to be that these apps are buggy and never get finished, so I figured it was worth posting an update. Another thousand commits later, Lion Reader supports every feature I care about, works...

I'm part of a small Discord server and thought it would be funny to make a Geoguessr-style game where you get presented with a random interesting message from the server and have to guess when, where and by who it was posted. How It Works The game works by running...