Musing: Consider a graph with "Performance, i.e. how much diverse valuable stuff you can accomplish" on the Y-axis, and "Time budget, i.e. how long you are allowed to operate as an agent with a computer" on the X-axis. Suppose that initially, frontier AIs are broadly superhuman when given very small time budgets, but subhuman when given large time budgets. That is, humans are better able to 'scale time budget' than AIs; humans are better at 'long-horizon agency skills.' This would be represented as a human line on the graph, and an AI line on the graph, that both go up and to the right, but that intersect: the AI line starts higher but has a lower slope. Suppose further that as AI progress continues, frontier AIs gradually get better at converting time budget into performance / scaling time budget / long-horizon agency skills. That is, the slope of the AI line increases. Perhaps it gradually gets closer to the theoretical limits, e.g. each time step the slope of the AI line gets 0.1% closer to the slope of the theoretical-limits-of-agency-skills line. (Which must of course be higher-slope than the human line; let's be conservative and say 2x higher.) What happens? See this vibecoded graph: tl;dr: As AI progress continues and AIs gradually get better at agency skills, the "crossover point" gets to larger and larger time horizons. That is, the crossover between the time budgets for which AIs outperform humans, and the time budgets for which AIs underperform, gets longer. (See the green line on the right graph, which is the actual data + some added noise) Specifically it gets longer exponentially... wait no, superexponentially! It only looks like an exponential initially. But as the slope of the AI line starts to get close to the slope of the human line, it starts to bend up a bit and then shoots up to infinity, and then it's over: AIs have better slope than humans; there is no longer any crossover point. AIs have "infinite horizon length" now. (Bonus: The dot

10+ years ago, I expected that self-driving trucks would be common on US highways by 2025, and self-driving would be having a large effect on the employment of long-haul truckers. In reality, self-driving trucks are still in testing on a limited set of highways and driving conditions. The industry still wants to hire more human long-haul truckers, and is officially expected to keep doing so for some time. I expected that long-distance trucking would have overtaken passenger cars as the "face" of self-driving vehicles; the thing that people argue about when they argue whether self-driving vehicles are safe enough, good or bad for society, etc. This has not happened. When people argue about self-driving vehicles, they argue about whether they want Waymo cars in their city. I expected that the trucking industry would shed a lot of workers, replacing them with self-driving trucks that don't need sleep, breaks, or drug testing. I expected that this would be an vivid early example of mass job loss to AI; and in turn that this would motivate more political interest in UBI. This, too, has not happened. (I certainly did not expect that the trucking industry in 2025 would be much more disrupted by anti-immigrant politics than by self-driving technology.)

theory: a huge part of having a good social life is just taking social bids whenever they become available. examples of social bids both large and small include: deciding whether to join your friends on a roadtrip; getting to know someone you just met; getting to better know someone you bump into occasionally but usually never talk to; standing in line, seeing something amusing, and having the option to point this out to another stranger in line; saying something funny in a group conversation; following up over text with someone after meeting them; flirting; cold emailing someone on the internet; catching up with a friend. there are a variety of reasons why we might end up not taking social bids. if you don't have the social ability to notice opportunities to take bids, you might miss bids that you could take. if you force yourself to take bids without the requisite social ability, and end up taking bids which you incorrectly believe to exist, you might act in ways that people find weird, and burn potential connections, or intrude on people. if you are really tired or low-bandwidth or depressed or stressed, you will not want to take bids, because taking bids requires quite a lot of activation energy investment. if you are really focused on something else, like a research project, you will not take bids, because they distract your focus. i find that when i make more successful social bids, i feel a greater sense of community and connection, and often feel happier too; and conversely, when i don't, i feel isolated. my most common reason for not making more social bids is often just being super tired or busy. also, i feel like people around you generally mirror the kind of energy you bring to social bids; if you are super enthusiastic and friendly about taking social bids, then other people will also treat your social bids similarly. if you are having an amazing time meeting lots of new cool people, then other people will be more excited to meet you and willing to h

Back in 2021 when image generators like DALL-E arrived, I was more astonished than most that they could turn descriptions into realistic pictures. For anyone who’s tried creating a convincing fake photograph themselves by piecing together and adjusting existing photographs using software like Adobe Photoshop (without AI tools), will find it virtually impossible to do. Merely producing convincing lighting (with plausible colour temperature and illumination/shadows) is very difficult. Let alone the ray-tracing required to create a photorealistic 3D view from an arbitrary angle, which almost no human can do manually with such software; at best a skilled artist could take hours to produce a semi-realistic picture of a simple scene. For AI to be able to do this convincingly, it surely needs a sophisticated understanding of the 3D world (rotation, perspective, lighting, etc - ideally full ray-tracing, plus other physics and pragmatics), which is (almost?) impossible to infer just from a load of photographs (ie 2D images) with captions/descriptions. Indeed I heard that neural networks intrinsically can’t infer these kinds of complex mathematical transformations; also that, as their output is a superposition of their training data (existing photographs), they can’t create arbitrary scenes either. So how do they manage to do all of this so well, or at all? (Quite aside from the near-miracle of inferring the very oblique relationship between training images and text descriptions of them.) Oddly I haven’t seen anyone raise this question, let alone provide a satisfactory answer.

There are a number of implicit concepts I have in my head that seem so obvious that I don't even bother verbalizing them. At least, until it's brought to my attention other people don't share these concepts. It didn't feel like a big revelation at the time I learned the concept, just a formalization of something that's extremely obvious. And yet other people don't have those intuitions, so perhaps this is pretty non-obvious in reality. Here’s a short, non-exhaustive list: * Intermediate Value Theorem * Net Present Value * Differentiable functions are locally linear * Theory of mind * Grice’s maxims If you have not heard any of these ideas before, I highly recommend you look them up! Most *likely*, they will seem obvious to you. You might already know those concepts by a different name, or they’re already integrated enough into your worldview without a definitive name. However, many people appear to lack some of these concepts, and it’s possible you’re one of them. As a test: for every idea in the above list, can you think of a nontrivial real example of a dispute where one or both parties in an intellectual disagreement likely failed to model this concept? If not, you might be missing something about each idea!

Epistemic status: puzzling something out, very uncertain, optimizing for epistemic legibility so I’m easy to argue with. All specific numbers are ass pulls]   In my ideal world, the anti-X-risk political ecosystem has an abundance of obviously high quality candidates. This doesn’t appear to be on the table. Adjusting for realism, my favorite is probably “have a wide top of funnel for AI safety candidates, narrow sharply as they have time to demonstrate traits we can judge them on”. But this depends on having an ecosystem with good judgement, and I’m not sure that that’s actually realistic.  I think it’s probably pretty easy to identify the top political candidates. These are the people like Alex Bores, who have a track record of getting hard legislation through their legislature and leave public evidence of strongly understanding the issue. If I had to put a number on it I’d count us as indescribably lucky to have 5% of candidates be this obviously good. It’s also pretty easy to identify the worst candidates, by track record or by the financial support of pro-AI PACs. Let’s optimistically put this at the bottom 50% of candidates. I don’t expect getting the obviously great 5% elected/appointed to be enough to win on AI safety issues. I also low-confidence expect supporting the middle 45% uniformly to be worse than doing nothing. And not just because some of them are lying- but because useful legislation is such a narrow target, and lots of people mean well without having the skill to actually be helpful. There’s also the negative externalities of crowding out better candidates. Say we need the 80-95th percentile candidates to win. A candidate who competes for the same support, who is well meaning but less competent than the 80th percentile, is actively taking away from better candidates.*  [*You can come up with math where this is okay if resources are abundant and the lesser candidates are merely less good rather than actively bad, but I expect resources to b

https://www.reuters.com/world/china/how-china-built-its-manhattan-project-rival-west-ai-chips-2025-12-17/  'Summary * Shenzhen team completed a working prototype of a EUV machine in early 2025, sources say * The lithography machine, built by former ASML engineers, fills a factory floor, sources say * China's EUV machine is undergoing testing, and has not produced working chips, sources say * Government is targeting 2028 for working chips, but sources say 2030 is more likely'