The important thing for alignment work isn't the median prediction; if we had an alignment solution just by then, we'd have a 50% chance of dying from that lack.

I think the biggest takeaway is that nobody has a very precise and reliable prediction, so if we want to have good alignment plans in advance of AGI, we'd better get cracking.

I think Daniel's estimate does include a pretty specific and plausible model of a path to AGI, so I take his the most seriously. My model of possible AGI architectures requires even less compute than his, but I think the Hofstadter principle applies to AGI development if not compute progress.

Estimates in the absence of gears-level models of AGI seem much more uncertain, which might be why Ajeya and Ege's have much wider distributions.

Your observations have a methodological flaw: people you know don't react better when you look nice because they know it's still you. Their reactions won't fluctuate with your daily appearance because they average their impression of you. Strangers' might; but some of it is also how you act differently based on how people treat you on-average, which makes appearance a subtle but longer term effect.

Speaking of which: how you act is more important. How you dress habitually does matter (but differently for different subcultures/ingroups). And good personal grooming (clean clothes and hair, haircut that fits your desired role) is easy, so you're shooting yourself in the foot if you don't bother with that.

There's a lot to it, but it's worth knowing the basics of how your habitual manner of interaction affects people and their reactions to you. Steve Byrnes' Making yourself small  is very insightful for the basics of social interaction dynamics. Just becoming a little conscious of how you're interacting with people goes a long way. I know that's not what you're asking, but could fit your final "Or am I missing something?"

This seems interesting and important. But I'm not really understanding your graph. How do I interpret action confidence? And what are the color variations portraying? Maybe editing in a legend would be useful to others too.

Marvelous! This makes more sense of the culture than the books do.

I haven't done much oppositional reading, but I enjoy oppositional watching. Movies need a lot more help than most books toward making sense. I'd tell you my theory of Sith ideology, but it's embarrassing to even have theories about a series made mostly for kids.

I was going to comment on the apparant deathism of the culture, which has always bothered me. Their cautious low level of interventions is a bit easier to explain, but the books don't bother to do it.

How about this: Their nonintervention is some remnant of an alignment that didn't allow them to intervene directly to influence humans, as a safety measure? And so special circumstance is only little efforts that are pursued by the very few Culture humans who care, aided by the few Minds that humor them.

I've been excited about pitching AGI x-risks to conservatives since seeing the great outreach work and writup from AE Studios, Making a conservative case for alignment.

My fervant hope is that we somehow avoid making this a politically polarized issue. I fear that polarization easily overwhelms reason, and is one of the few ways the public could fail to appreciate the dreadful, simple logic in time to be of any help.

We are building new gods in the hope that they will love us.

You may laugh as though this were impossible or recoil in fear if you know it is.

But it is what we're doing.

This eclipses all other efforts to navigate those eldritch forces.

Will our new gods love us, or at least obey us? If they obey, will their mortal masters use their power for their brethren, or to create strange new worlds?

This piece is amazing. Thank you. I absolutely love the framing and agree with the analysis of the situation - only the action recommendations need to be updated

Like most incisive analyses of the current historical situation, this piece is hugely incomplete, particularly because it does not take account for likely progress in AI.

I am in love with my ebike. I may get less exercise total, but I get a lot more joy from a lot more time outdoors. I feel a stronger connection to place and culture, because the bike allows going slow and looking around. When there's traffic or you're going fast, your attention will and should be mostly on that, but you can stop or go slow as much as you want to lok around. Rubbernecking in a car can feel rude or dangerous; on a bike it's easy and fun.

Having a big convenient cargo carrier and places to secure collapsible bags for extra space means you can do most errands by bike instead of car.

Increased time on a bike sounds risky, but here's a weird study: there were more closed head injuries recorded per mile travelled in a car vs. a bike. I didn't catch the ref and I realize this is absolutely astounding. The ones on bikes are probably more severe, but I think it's not fully appreciated how much even minor car accidents mess your brain up.

I do recommend biking as though no car will ever notice you by default, but realize that's not really an option in dense cities with a lot of traffic. For smaller towns with less traffic and more side streets and bike lanes (Boulder is an extreme but I'm also biking in Traverse City, a smaller and much less bike-friendly town) there's no comparison in levels of joy and fun to driving.

What I make of those studies is that stimulating the thalamus activates the whole corticothalemic loop. The non-specific or activating nuclei of the thalamus switch on matched areas of cortex. The thalamus has a powerful regulatory role, but it's not making decisions, it's enforcing them. The government that sits in DC is making decisions and that's why we call it the seat of the government. Your metaphor simply does not go through and it makes you sound confused.

The government's decisions are influenced from elsewhere, and they are enacted elsewhere. But the thalamist's role is much less like the Congress or Senate and much more like the people who enact and enforce the decisions made by those governing bodies. The decisions about what becomes conscious are made elsewhere, in the conjunction of the cortex and basal ganglia. Decisions about whether to be conscious or unconscious or made in subthalamic nuclei, and again only enforced or enacted by the thalamus.

There you said the thalamus is where consciousness is happening. That is just flat wrong. It's a system phenomenon. Trending towards statements like that is why it's a mistake to say any place is the seat of consciousness; it leads to very wrong conceptions and statements like that.

Government largely happens in DC. Consciousness largely happens throughout thalamocortical loops.

The reason I thought this is worth mentioning is that talking about a seat of consciousness confuses the whole phenomenon of consciousness. It implies that consciousness is some little add-on happening in some little corner of the brain, when that's not right at all; consciousness is a highly complex phenomena involving much of the brain's higher functions.

I appreciate you saying that the 25th percentile timeline might be more important. I think that's right and underappreciated.

One of your recent (excellent) posts also made me notice that AGI timelines probably aren't normally distributed. Breakthroughs, other large turns of events, or large theoretical misunderstandings at this point probably play a large role, and there are probably only a very few of those that will hit. Small unpredictable events that create normal distributions will play a lesser role.

I don't know how you'd characterize that mathematically, but I don't think it's right to assume it's normally distributed, or even close.

Back to your comment on the 25th percentile being important: I think there's a common error where people round to the median and then think "ok, that's probably when we need to have alignment/strategy figured out." You'd really want to have it at least somewhat ready far earlier.

That's both in case it's on the earlier side of the predicted distribution, and because alignment theory and practice need to be ready far enough in advance of game time to have diffused and be implemented for the first takeover-capable model.

I've been thinking of writing a post called something like "why are so few people frantic about alignment?" making those points. Stated timeline distributions don't seem to match mood IMO and I'm trying to figure out why. I realize that part of it is a very reasonable "we'll figure it out when/if we get there." And perhaps others share my emotional dissociation from my intellectual expectations. But maybe we should all be a bit more frantic. I'd like some more halfassed alignment solutions in play and under discussion right now. The 80/20 rule probably applies here.

Okay fine, I'll engage a little. I do love this shit, even though I try not to spend time on it because it's a mess with little payoff (unless the aforementioned debate over AI consciousness starts to seem relevant to our odds of survival - which it well might).

I don't think your DC as the seat of government metaphor goes through. DC is indeed the seat of government. The thalamus isn't in charge of consciousness, it's just a valve (but far more sophisticated; an arena of competition) that someone else turns: the cortex and basal ganglia, in elaborate collaboration. The thalamus is the mechanism by which their decisions are enforced; it doesn't seem to play a large role in deciding what's attended.