Quick Takes

Has anyone tried to make a sort of standardized test for collaborative rationality/coordination skills (double-crux, ITT, things like in this post)? It seems to me like that + a sort of badges system on a browser extension[1] solves problems where both people would coordinate and use a better strategy if they knew the other person was able to. [1] allows for use on more than just this site, also I don't know how I would feel about this being an Official LW Thing

Would we be less likely to go extinct if OpenAI forcibly took over Anthropic? (a possible hypothetical: a combined effort of OpenAI and the US government forces a merger). My take: no. Main reasons: * I trust OpenAI leadership less than Anthropic leadership * Race dynamics would not improve due to the other players. I judge the creation of Anthropic as retrospectively a good choice. At the time I was negative and have become less so. Similarly, would we be less likely to go extinct if Alphabet forcibly took over OpenAI and Anthropic? My take: yes. Main reasons: * Substantial improvement in race dynamics. * Alphabet has less corporate need for urgent AI advances. * Alphabet has a better security posture. I judge the creation of OpenAI as retrospectively a bad choice. At the time I was negative and have become more so.

A thing I imagine some people miss about near-term non-X-risk impacts of gippities: If 95% of some task is automatable, that doesn't necessarily mean you can speed it up 20x using automation. That may sound strange, but consider this: there's a lot of hidden value in humans having context loaded up. If the non-automatable 5% is really important, then you still want the human doing that 5% well. For the human to do it well, the human may have to have deeply reviewed many parts of the 95%. For example, even if some judge's ruling will only directly warrant some obvious automated next response, that doesn't mean the human can just skip reading the ruling; some aspects of it may inform the deeper legal strategy. Or something. Similarly, if the human needs to make many of the deepest architectural choices in a big software project, the human may have to be well familiar with the constraints of many specific elements of the project, even of the immediate functionality of those elements could easily be implemented by gippity coding. (None of this strongly implies there won't be some huge effects from gippities, and none of this bears much on actual AGI.)

Did anyone predict the "LLM psychosis" / "LLM mania" / that thing that happens where people and LLMs do a folie-a-deux beforehand? I believe no one did, it's completely the result of actual interactions with the world, but maybe someone has a reasonable claim.

AI X risk is almost perfectly designed to oneshot people with OCD. The combination of absurdly high stakes, uncertainty, and a community dedicated to discussing it 24/7 with inquiry considered a virtue is tailor made to trigger one's OCD. Further, it's not exactly a subject that therapists would take seriously though, for better or for worse (and definitely for worse), I imagine that's starting to change. I had wondered for a while why I responded so much worse on a mental level than basically anyone else and, upon being diagnosed with Obsessive Compulsive Disorder, finally have an explanation. The typical treatment for OCD beyond medication is quite effective and called Exposure Response Therapy. With it, you practice not partaking in your compulsions and noticing that nothing bad happened. This sort of thing would work for someone obsessed with cleanliness but not someone who thinks there's a 30% chance that we will all be dead in 10 years. That being said, my dealing with X risk is very characteristic of OCD in a way which makes me think I could respond well to therapy, however the treatment would likely, though not necessarily, be more bespoke than is typical. In terms of how I was specifically affected: I would run and rerun the same anti-doom arguments in my head over and over, and was literally unable to stop thinking about it to the point of being dysfunctional. I'm better now, though not where I'd like to be. I was wondering if there is anyone here who with OCD who would be willing to talk with me about their experience and what sorts of treatments they did to get better or put me in contact with someone who could. I would be willing to pay up to 1000$ for this, though given the unverifiable nature I'm not sure how I would go about this.

Someone wrote a "contra" post for my post! I'm a real rationalist blogger now! At least until I start thinking I need to achieve some higher goal like writing something actually good. But I sure will ride this high for the next week or so. In other news, I attended and perhaps slightly organized a small 1-day LWCW-inspired unconference in Espoo, Finland. I was, as usual, facilitating circling and hotseat. Other interesting stuff occurred too. The experience for me was quasi-trancendental, personality-wise. Or perhaps this simply continues my fake enlightenment arc that's been having across the past week or two. In any case, this is the stuff I crave. On an unrelated note, optimization is the process of extracting fun from a something. Or perhaps fun is the process of optimizing it out of the world. "All models are wrong, some are useful" and this one is hopefully useless and thus a great source of fun until it gets useful.

Here's a thought I might expand on but that I'll just mention quickly now. I feel there's a gap in the AI Safety field when it comes to automated alignment science. There's a lot of talk about verification of individual knowledge claims and similar yet it doesn't feel like anyone is bringing principles from metascience and similar? If science is a collective process of generation and verification why are we not talking about things like generating distributed fault tolerance algorithms or ways of collectively verifying knowledge so that we know that any individual piece of work is right? There's an existent field of meta science and whenever I see posts about automated alignment science I never see any discussion of this? Where are my Michael Nielsen references? Where are the ideas about proof burden and verification and thinking about this from a philosophy of science perspective? See the following posts as examples, I don't see anyone citing or mentioning much metascience stuff. (I'm not saying these are bad posts, I want to point out that there's a missing mood): * https://www.lesswrong.com/posts/qdhyrN4uKwBAftmQx/there-should-be-usd100m-grants-to-automate-ai-safety * https://www.lesswrong.com/posts/z4FvJigv3c8sZgaKZ/will-we-get-automated-alignment-research-before-an-ai * https://www.lesswrong.com/posts/FqpAPC48CzAtvfx5C/automating-ai-safety-what-we-can-do-today * https://www.lesswrong.com/posts/nJcuj4rtuefeTRFHp/can-we-safely-automate-alignment-research Where exactly in science do we see a 1 on 1 verification regime where a singular human verifies the fact that something is true or false? Byzantine fault tolerance is a distributed scheme if you want convergent safety properties it makes more sense to look at systems with less individual failure points? (Some stuff that examplify what I think should be looked at: ) * A Vision of Metascience - A generally very good article on metascience and what science could be * Knowledge Lab - A lab studying vari