My answer is going to be unsatisfying - entirely vibes. While there are still significant sections of the populace that have left-over affection for anything that looks like the Civil Rights movement due to how valorized that movement is and how much change it affected, this is seriously waning. The non-effectiveness of movements that just copy the aesthetics are slowly making them look more like cargo-cults that copy the form but without an understanding of the substance that made them successful.
As more people dismiss protestors as performance without substance, protests start getting more awful to get anyone's attention. Destroying social value and public goods for a cause no one else cares about grows increasingly irksome. When major lawlessness threatens people and sets fire to city blocks in the name of activism the good will drains away pretty rapidly. Now the cargo cults are just destroying stuff without any path to how that's supposed to make things better.
It's an ongoing change. We're only seeing the start of it. But IMO its pretty undeniable that a decent percentage of the population thinks of activists as default harmful, and a preference cascade is just over the horizon.
Seems like the best option. Very frustrating to me, who wants to keep on to all four of his current projects. :( But coming to realize this is a terrible idea.
I worry that it's a fundamental enough part of being an intelligent agent that losing this capability will make us less human. Similar to the WALL-E humans that had lost the ability to walk due to never walking. They were legitimately lesser beings due to that. :(
AI is Immersed Mindset On Demand
At The Bayesian Conspiracy we recently spoke with Zoe about pop-up cities and goal-focused group houses (episode coming next week). One of the great things about these settings is that being immersed in your work keeps you in a mental frame where related self-propagating thought patterns are kept alive in your brain and stay processing in the background all the time.
Much of our brain power is ambient. It takes a while to shut down thought patterns that have been oscillating for a while and recruit those neurons into new patterns. In fact I think it’s worse than this - I think it can be very easy to restart patterns every morning that you’ve been using for days, and much harder to restart ones you haven’t used in days, or weeks, or months. Thus the phenomenon of taking several days to get running again when you return from a long vacation.
This is extremely noticeable when trying to write fiction, and it should be true of any major project. Your brain can’t turn deep focus on and off. Your brain is a steam locomotive, with dozens of reinforcing thought oscillations. When you jump off it and run in a different direction for a while that’s fine, you can run back to the tracks and get back to stoking the engine with a prize from your expedition in your back pocket. But the more you slip off for side quests, and the longer you stay away, the more the train tracks will start to re-lay themselves, so the locomotive realigns to follow you, and your previous goal is left behind.
Attention is all you need. Attention is all you have. Everything follows your attention over a long time scale.
AIs Can Imagine It For You Wholesale
Recently we were asked what question we’d would put in a general survey about The Bayesian Conspiracy. I was stumped. I had nothing. My focus is split among 4+ projects, I am not immersed, I would have to sit down and stew on this.
Claude gave a fantastic answer is a split second. Several great answers, actually. It's not that it’s incredibly creative, because it’s not. But it’s great at the basic level of creativity one gets from an immersed mindset. The mental framework that would require ongoing dedication from a human (on the level of daily focus across weeks) is what the AI can deliver on demand.
Now that we have the basic creative output, we can choose among Claude’s suggestions and tighten them up.
I don’t like this. It feels dangerous. It’s too tempting. Being deeply focused on just one topic for weeks means giving up on a LOT of other things that I don’t want to give up on. But if I compensate by relying on this tool I worry I’ll stop going into deep focus entirely, and just lose the knack for it over time. Why wouldn’t most people do so, if they could? How many humans in the present day can memorize and recite long orations? We don’t need to now that we have written language. Per Plato, have we not leaned close to becoming “hearers of many things”, [appearing] as though they were all-knowing, but to actually be learners of nothing.
Soon we won’t even have to think up our own thoughts. Ten years from now [1]the AIs will be doing most of the things we think of as creative-generation and humans will mostly be the Deciders, picking among the options presented to get at the true essence of what we want. A cool premise for a sci fi world, but not a world I would choose.
(this is a mirror of a blog post at DeathIsBad)
Given the standard anti-doom caveats
Definitely not wrong, the petitions almost certainly won't change anything. Change.org is not where one goes to actually change things.
I had a quasi-romantic relationship with a fictional character that lived in my head during my worst year, in college. I could sometimes even "see" him. I knew he wasn't real. It did help me out during the darkest times. Probably woulda been even better to be able to have chat conversations that were run by an AI. And I did outgrow that in a few months, when life got better.
So, basically, this sounds great and I love this perspective. Thank you.
answer:
2 of 7
I have a secret desire for this to become real which I fear may destroy me and/or everything I know.
The graph image is broken. Does anyone have a copy of the image file? I remember what it looked like, and it was super-useful for demonstrating the concept.
I've been notified that this post was nominated as a finalist for the Less Wrong 2023 Review! This is fantastic news, and I'm deeply honored! As part of the notification I was encouraged to write a self-review, with some example prompts like "Do you still endorse this?" and "What further work do you think should be done exploring the ideas here?”
Fiction is pretty Out Of Distribution for Less Wrong posts. I almost didn’t post it, because what is the point? I mean that literally... what IS the point of fiction on Less Wrong? Most often it’s to help demonstrate a different point that’s the real purpose of the post. That doesn’t make for good fiction, any more than “Bobby has three apples, John steals two of his apples, now Bobby has one apple” makes for good fiction. Sure there’s action and maybe even emotion, but the goal was to teach math.
The goal of a Less Wrong post is to teach. To impart some intellectual insight or real-world observation or new (or updated) technique. That is not the goal of fiction. The goal of fiction is to create an emotion experience or to entertain. Truly skilled authors can achieve both a LW and a Fiction goal in the same work, but I am not that skilled. My goal is a story that entertains and creates emotion before anything else, and for me that means sacrificing the ability for a story to teach rationality.
When I workshopped the first draft of the story I was asked ‘Why would the AI work in that way? It’s impossible for it to have come to such a utility function on accident, and literally no one has the motivation to make it have this utility function.’ ‘Why do these two aspects of the AI act like this, that’s not how a functional entity would act given X-Z.’ The answer to all questions of this type were “Yes, I know. This isn’t meant to be a realistic portrayal of what could happen. This is supposed to be fun. And this personality, and these problems that this personally runs into, are actually really fun! Can you point me to the parts that reduce fun or emotion? I don’t care about the story actually being rational.”
This puts the story directly at odds with what a Less Wrong post is supposed to be. I had severe reservations about posting it. But maybe fiction can have good secondary effects in the rationalist world even if its not at all rational. Terminator 2 is extremely non-rational, and IIRC is often cited by Eliezer as a particularly egregious movie because it primed people to think of AIs as dumb things that would use robot-soldiers we can defeat. It sacrificed rational depiction of AI danger on the alter of making an exciting story that an average person could be a hero within.
And yet, T2 probably has done more to bring the mass of the populace on board with worrying about AI risk than any other single thing in our lifetimes. It was easy to understand, very emotionally compelling (which gives it a long lifespan in our memory and a lot of weight in our pre-deliberative intuitions), and extremely popular (which spread its impact wide and gave everyone a common cultural touchstone). Despite being non-rational (or anti-rational) it was a huge boon to the cause of rationality.
So maybe it’s ok to post a story that isn’t actually rational and doesn’t do any of the things a Less Wrong post is supposed to do. Maybe just sharing a fun story on its own can be worth it if it plants an emotional seed that aligns with the rationalist cause. I do think The Real Fanfic is a fun story. I also think it contains the seed of “even a god that superficially looks like us and seems to have our interests at heart can be incredibly creepy and bad, actually, for reasons that aren’t at all obvious.”
So, if my goal is to give something fun to my fellow rats, this is the best place to do so. And if it can advance rationality in some small way it’s probably even worth doing so, despite being a very non-central post. I figured it was worth a shot. A decent amount of post karma, plus a nomination to the 2023 Review, leads me to suspect some people agreed.
So “Do I still endorse this?” It’s story I’m still proud of. :) Whether you enjoyed it or very much didn’t, I’m excited to read any review.