(Disclaimer: This is my personal opinion, not that of any movement or organization.)

This post aims to show that, over the next decade, it is quite likely that most democratic Western countries will become fascist dictatorships - this is not a tail risk, but the most likely overall outcome. Politics is not a typical LessWrong topic, and for good reason:

1. it tends to impair clear thinking;
2. most well-known political issues are not neglected;
3. most political "debates" are simply people yelling at each other online; neither saying anything new, nor even really trying to persuade the opposition.

However, like the COVID pandemic, it seems like this particular trend will be so impactful and so disruptive to ordinary Western life that it will be important to be aware of it, factor it into plans,...

5.1 Post summary / Table of contents

This is the 5th of a series of 8 blog posts, which I’m serializing weekly. (Or email or DM me if you want to read the whole thing right now.)

Dissociative Identity Disorder (DID) (previously known as “Multiple Personality Disorder”) involves a person having multiple “alters” (alternate identities), with different preferences and (in some cases) different names. A DID diagnosis also requires some nonzero amount of “inter-identity amnesia”, where an alter cannot recall events that occurred when a different alter was active. For example, DSM-V talks about patients “coming to” on a beach with no recollection of how they got there.

Anyway, just like trance in the previous post, DID was one of those things that I unthinkingly assumed was vaguely fictional for most of...

It's been a busy season at the Nucleic Acid Observatory, and we have a lot to share since our last update. As always, If anything here is particularly interesting or if you’re working on similar problems, please reach out!

Wastewater Sequencing

We performed an initial analysis of untargeted sequencing data from aggregated airplane lavatory waste and municipal treatment plant influent that we collected and processed during our Fall 2023 partnership with CDC’s Traveler-based Genomic Surveillance program and Ginkgo Biosecurity. We've now analyzed viral abundance and diversity in sequencing data across multiple sample types and wastewater-processing and sequencing protocols. Next steps include further investigating how protocol and sample-type affect specific viruses and bacteria, as well as understanding pathogen temporal dynamics seen in airport versus treatment-plant samples.

We have continued to work...

Dario Amodei is thinking about the potential. The result is a mostly good essay called Machines of Loving Grace, outlining what can be done with ‘powerful AI’ if we had years of what was otherwise relative normality to exploit it in several key domains, and we avoided negative outcomes and solved the control and alignment problems. As he notes, a lot of pretty great things would then be super doable.

Anthropic also offers us improvements to its Responsible Scaling Policy (RSP, or what SB 1047 called an SSP). Still much left to do, but a clear step forward there.

Daniel Kokotajlo and Dean Ball have teamed up on an op-ed for Time on the need for greater regulatory transparency. It’s very good.

Also, it’s worth checking out the Truth Terminal...

It’s monthly roundup time again, and it’s happily election-free.

Thinking About the Roman Empire’s Approval Rating

Propaganda works, ancient empires edition. This includes the Roman Republic being less popular than the Roman Empire and people approving of Sparta, whereas Persia and Carthage get left behind. They’re no FDA.

Polling USA: Net Favorable Opinion Of:

Ancient Athens: +44%

Roman Empire: +30%

Ancient Sparta: +23%

Roman Republican: +26%

Carthage: +13%

Holy Roman Empire: +7%

Persian Empire: +1%

Visigoths: -7%

Huns: -29%

YouGov / June 6, 2024 / n=2205

The Five Star Problem

What do we do about all 5-star ratings collapsing the way Peter describes here?

Peter Wildeford: TBH I am pretty annoyed that when I rate stuff the options are:

* “5 stars – everything was good enough I guess”

* “4 stars – there was a serious problem”

* “1-3 stars – I almost died”

I can’t

I think and talk a lot about the risks of powerful AI. The company I’m the CEO of, Anthropic, does a lot of research on how to reduce these risks. Because of this, people sometimes draw the conclusion that I’m a pessimist or “doomer” who thinks AI will be mostly bad or dangerous. I don’t think that at all. In fact, one of my main reasons for focusing on risks is that they’re the only thing standing between us and what I see as a fundamentally positive future. I think that most people are underestimating just how radical the upside of AI could be, just as I think most people are underestimating how bad the risks could be.

In this essay I try to sketch out what that

the following is motivated by:

• Self-Improvement or Shiny Distraction: Why Less Wrong is anti-Instrumental Rationality
• The Martial Art of Rationality

I've been a long time lurker on Less Wrong and I've noticed the recurring criticism that despite its focus on rationality, the community lacks structured training to develop practical rationality skills. Eliezer Yudkowsky talks rationality as a martial art, because it's something that can be trained and refined through deliberate practice. But where is our dojo?

A model that comes to mind is a website like LeetCode, where programmers can solve coding challenges, share solutions, and see how others approach the same problems. LeetCode can sometimes encourage overfitting to specific problem types so it's not a perfect analogy. The community driven aspect would interesting to me as you can see how...

How can we make many humans who are very good at solving difficult problems?

Summary (table of made-up numbers)

I made up the made-up numbers in this table of made-up numbers; therefore, the numbers in this table of made-up numbers are made-up numbers.

Call to action

If you have a shitload of money, there are some projects you can give money to that would make supergenius humans on demand happen faster. If you have a fuckton of money, there are projects whose creation you could fund that would greatly accelerate this technology.

If you're young and smart, or are already an expert in either stem cell / reproductive biology, biotech, or anything related to brain-computer interfaces, there are some projects you could work on.

If neither, think hard, maybe I missed something.

You can...

Alright, I have a question stemming from TurnTrout's post on Reward is not the optimization target, where he argues that the premises that are required to get to the conclusion of reward being the optimization target are so narrowly applicable as to not apply to future RL AIs as they gain more and more power:

https://www.lesswrong.com/posts/pdaGN6pQyQarFHXF4/reward-is-not-the-optimization-target#When_is_reward_the_optimization_target_of_the_agent_

But @gwern argued with Turntrout that reward is in fact the optimization target for a broad range of RL algorithms:

https://www.lesswrong.com/posts/ttmmKDTkzuum3fftG/#sdCdLw3ggRxYik385

https://www.lesswrong.com/posts/nmxzr2zsjNtjaHh7x/actually-othello-gpt-has-a-linear-emergent-world#Tdo7S62iaYwfBCFxL

So my question is are there known results, ideally proofs, but I can accept empirical studies if necessary that show when RL algorithms treat the reward function as an optimization target?

And how narrow is the space of RL algorithms that don't optimize for the reward function?

A good answer will link to results known in...