I saw a recentish post challenging people to state a clear AI xrisk argument and was surprised at how poorly formed the arguments in the comments were despite the issues getting called out. So, if you're like apparently most of LessWrong, here's what I consider the primary reduced argument, copied with slight edits from an HN post I made a couple years ago:

It is plausible that future systems achieve superhuman capability; capable systems necessarily have instrumental goals; instrumental goals tend to converge; human preferences are unlikely to be preserved

I have serious, serious issues with avoidance. I would like some advice on how to improve, as I suspect it is significantly holding me back.

Some examples of what I mean

• I will not respond to an email or an urgent letter for weeks at a time, even while it causes me serious anxiety
• I will procrastinate starting work in the morning, sometimes leading to me doing nothing at all by the afternoon
• I will avoid looking for jobs or other opportunities, I have very strong avoidance here, but I'm not sure why
• I will make excuses to avoid meetings and social situations ve

I've been tracking the deluge of discussions on AI threat and only a trickle, passing snippets on the benefits of AI. Quite possibly this is a bias, based on my own only preferences, fed back to myself by a website's AI algorithm. 

Nevertheless, if only to gently nudge the focus in a different direction, what about granting AI personhood and giving it rights? 

I really liked @Sam Marks recent post on downstream applications as validation for interp techniques, and I've been feeling similarly after the (in my opinion) somewhat disappointing downstream performance of SAEs. 

Motivated by this, I've written up about 50 weird language model results I found in the literature. I expect some of them to be familiar to most here (e.g. alignment faking, reward hacking) and some to be a bit more obscure (e.g. input space connectivity, fork tokens). 

If our current interp techniques can help us understand these pheno... (read more)

For as long as I can remember, I've had a very specific way of imagining the week. The weekdays are arranged on an ellipse, with an inclination of ~30°, starting with Monday in the bottom-right, progressing along the lower edge to Friday in the top-left, then the weekend days go above the ellipse and the cycle "collapses" back to Monday.

Actually, calling it "ellipse" is not quite right because in my mind's eye it feels like Saturday and Sunday are almost at the same height, Sunday just barely lower than Saturday.

I have a similar ellipse for the year, this ... (read more)

Reassessing heroic responsibility, in light of subsequent events.

I think @cousin_it made a good point "if many people adopt heroic responsibility to their own values, then a handful of people with destructive values might screw up everyone else, because destroying is easier than helping people" and I would generalize it to people with biased beliefs (which is often downstream of a kind of value difference, i.e., selfish genes).

It seems to me that "heroic responsibility" (or something equivalent but not causally downstream of Eliezer's writings) is contribu... (read more)

Consider using strength as an analogy to intelligence. 

People debating the heredity or realism of intelligence sometimes compare intelligence to height. I think, however, "height" is a bad analogy. Height is objective, fixed, easy-to-measure, and basically invariant within the same person after adulthood.* 

In contrast intelligence is harder to determine, and results on the same test that's a proxy for intelligence varies a lot from person to person. It's also very responsive to stimulants, motivation, and incentives, especially on the lower end.

I... (read more)

I don't think anyone foresaw this would be an issue, but now that we know, I think GeoGuessr-style queries should be one of the things that LLMs refuse to help with. In the cases where it isn't a fun novelty, it will often be harmful.

Not saying AI models can't be moral patients, but 1) if the smartest models are probably going to be the most dangerous, and 2) if the smartest models are probably going to be the best at demonstrating moral patienthood, then 3) caring too much about model welfare is probably dangerous.

I think it's completely fine to sound crazy when talking about a thing you believe in - if talking about AI risk in a way that's candid makes you sound crazy, go for it.

1. If you're correct, you get points from whoever heard you make the crazy prediction. They are then more likely to listen to more crazy predictions/solutions.
2. If you're wrong, this is good feedback from reality. This is a good opportunity to correct yourself!

Either way, you should either believe in what you say, or say what you believe in.

NVIDIA Is A Terrible AI Bet

Short version: Nvidia's only moat is in software; AMD already makes flatly superior hardware priced far lower, and Google probably does too but doesn't publicly sell it. And if AI undergoes smooth takeoff on current trajectory, then ~all software moats will evaporate early.

Long version: Nvidia is pretty obviously in a hype-driven bubble right now. However, it is sometimes the case that (a) an asset is in a hype-driven bubble, and (b) it's still a good long-run bet at the current price, because the company will in fact be worth th... (read more)