I'm upping my vote for this in the year-end review based on the latest Zizian murder. :/ https://twitter.com/LexerLux/status/1882438198296744130#

I have very mixed feelings about GlowFic which are a direct result of trying to read PlaneCrash.

Pro: they are a joy for the author, and gets an author to write many hundreds of thousands of words effortlessly, which is great when you want more words from an author. 

Con: the format is anti-conducive to narrative density. The joy is in creating any words, which is great for the author, but bad for audiences. Readers want a high engagement-per-word ratio.

For context, my two favorite works are HPMOR and Worth The Candle, of 660K and 1.6M words. I spent 16 months podcasting a read-through/analysis of Worth The Candle. I'm not shy about reading lots of words. But when I tried to do the same for PlaneCrash I stopped after 200k words. The problem was not the decision theory or the math, which I found interesting in the brief sections they came up. The problem was plot and character development. 200,000 words is two full novels. In  a single novel an author will typically build an entire world and get us to fall in love with the characters, throw them into conflict, build to a climax, drive at least one character through an entire character arc where they develop as a person, and bring an audience through emotional catharsis via a climax and resolution. Often with side-plots or supporting characters fleshed out as well. In 200,000 words this could be done TWICE (maybe 1.5 times if the books run long).

In 200,000 words of PlaneCrash we got through one major plot point and set up the next. It felt like as much action as you get roughly within 25k words normally. That's an order of magnitude more cost-per-payoff compared to any general-audience novel (including HPMOR). This is more than even I am willing to pay.

As Devon Erikson says, every word is a bid for the attention of the reader, it's a price the author is bidding up. They need to recoup that cost by delivering to the reader a greater amount of something the reader wants - enjoyment, excitement, insight, information, emotional release, whatever. In the GlowFic format, authors are primarily writing for their own enjoyment, and perhaps their onlooking friends. Mass audiences don't get the expected per-word payoff. 

I think a serious review of PlaneCrash such as this one should acknowledge that the narrative-to-wordcount ratio is way out of proportion to what most people will accept, and this is the major flaw of the piece.

Really annoying that that's not available on the app! Oliver's added the transcript in the main post now, thankfully. :)

I've been notified that this post was nominated as a finalist for the Less Wrong 2023 Review! This is fantastic news, and I'm deeply honored! As part of the notification I was encouraged to write a self-review, with some example prompts like "Do you still endorse this?" and "What further work do you think should be done exploring the ideas here?”

Fiction is pretty Out Of Distribution for Less Wrong posts. I almost didn’t post it, because what is the point? I mean that literally... what IS the point of fiction on Less Wrong? Most often it’s to help demonstrate a different point that’s the real purpose of the post. That doesn’t make for good fiction, any more than “Bobby has three apples, John steals two of his apples, now Bobby has one apple” makes for good fiction. Sure there’s action and maybe even emotion, but the goal was to teach math.

The goal of a Less Wrong post is to teach. To impart some intellectual insight or real-world observation or new (or updated) technique. That is not the goal of fiction. The goal of fiction is to create an emotion experience or to entertain. Truly skilled authors can achieve both a LW and a Fiction goal in the same work, but I am not that skilled. My goal is a story that entertains and creates emotion before anything else, and for me that means sacrificing the ability for a story to teach rationality.

When I workshopped the first draft of the story I was asked ‘Why would the AI work in that way? It’s impossible for it to have come to such a utility function on accident, and literally no one has the motivation to make it have this utility function.’ ‘Why do these two aspects of the AI act like this, that’s not how a functional entity would act given X-Z.’ The answer to all questions of this type were “Yes, I know. This isn’t meant to be a realistic portrayal of what could happen. This is supposed to be fun. And this personality, and these problems that this personally runs into, are actually really fun! Can you point me to the parts that reduce fun or emotion? I don’t care about the story actually being rational.”

This puts the story directly at odds with what a Less Wrong post is supposed to be. I had severe reservations about posting it. But maybe fiction can have good secondary effects in the rationalist world even if its not at all rational. Terminator 2 is extremely non-rational, and IIRC is often cited by Eliezer as a particularly egregious movie because it primed people to think of AIs as dumb things that would use robot-soldiers we can defeat. It sacrificed rational depiction of AI danger on the alter of making an exciting story that an average person could be a hero within.

And yet, T2 probably has done more to bring the mass of the populace on board with worrying about AI risk than any other single thing in our lifetimes. It was easy to understand, very emotionally compelling (which gives it a long lifespan in our memory and a lot of weight in our pre-deliberative intuitions), and extremely popular (which spread its impact wide and gave everyone a common cultural touchstone). Despite being non-rational (or anti-rational) it was a huge boon to the cause of rationality.

So maybe it’s ok to post a story that isn’t actually rational and doesn’t do any of the things a Less Wrong post is supposed to do. Maybe just sharing a fun story on its own can be worth it if it plants an emotional seed that aligns with the rationalist cause. I do think The Real Fanfic is a fun story. I also think it contains the seed of “even a god that superficially looks like us and seems to have our interests at heart can be incredibly creepy and bad, actually, for reasons that aren’t at all obvious.”

So, if my goal is to give something fun to my fellow rats, this is the best place to do so. And if it can advance rationality in some small way it’s probably even worth doing so, despite being a very non-central post. I figured it was worth a shot. A decent amount of post karma, plus a nomination to the 2023 Review, leads me to suspect some people agreed.

So “Do I still endorse this?” It’s story I’m still proud of. :) Whether you enjoyed it or very much didn’t, I’m excited to read any review.

My answer is going to be unsatisfying - entirely vibes. While there are still significant sections of the populace that have left-over affection for anything that looks like the Civil Rights movement due to how valorized that movement is and how much change it affected, this is seriously waning. The non-effectiveness of movements that just copy the aesthetics are slowly making them look more like cargo-cults that copy the form but without an understanding of the substance that made them successful.

As more people dismiss protestors as performance without substance, protests start getting more awful to get anyone's attention. Destroying social value and public goods for a cause no one else cares about grows increasingly irksome. When major lawlessness threatens people and sets fire to city blocks in the name of activism the good will drains away pretty rapidly. Now the cargo cults are just destroying stuff without any path to how that's supposed to make things better.

It's an ongoing change. We're only seeing the start of it. But IMO its pretty undeniable that a decent percentage of the population thinks of activists as default harmful, and a preference cascade is just over the horizon.

Seems like the best option. Very frustrating to me, who wants to keep on to all four of his current projects. :( But coming to realize this is a terrible idea.

I worry that it's a fundamental enough part of being an intelligent agent that losing this capability will make us less human. Similar to the WALL-E humans that had lost the ability to walk due to never walking. They were legitimately lesser beings due to that. :(

AI is Immersed Mindset On Demand

At The Bayesian Conspiracy we recently spoke with Zoe about pop-up cities and goal-focused group houses (episode coming next week). One of the great things about these settings is that being immersed in your work keeps you in a mental frame where related self-propagating thought patterns are kept alive in your brain and stay processing in the background all the time.

Much of our brain power is ambient. It takes a while to shut down thought patterns that have been oscillating for a while and recruit those neurons into new patterns. In fact I think it’s worse than this - I think it can be very easy to restart patterns every morning that you’ve been using for days, and much harder to restart ones you haven’t used in days, or weeks, or months. Thus the phenomenon of taking several days to get running again when you return from a long vacation.

This is extremely noticeable when trying to write fiction, and it should be true of any major project. Your brain can’t turn deep focus on and off. Your brain is a steam locomotive, with dozens of reinforcing thought oscillations. When you jump off it and run in a different direction for a while that’s fine, you can run back to the tracks and get back to stoking the engine with a prize from your expedition in your back pocket. But the more you slip off for side quests, and the longer you stay away, the more the train tracks will start to re-lay themselves, so the locomotive realigns to follow you, and your previous goal is left behind.

Attention is all you need. Attention is all you have. Everything follows your attention over a long time scale.

AIs Can Imagine It For You Wholesale

Recently we were asked what question we’d would put in a general survey about The Bayesian Conspiracy. I was stumped. I had nothing. My focus is split among 4+ projects, I am not immersed, I would have to sit down and stew on this.

Claude gave a fantastic answer is a split second. Several great answers, actually. It's not that it’s incredibly creative, because it’s not. But it’s great at the basic level of creativity one gets from an immersed mindset. The mental framework that would require ongoing dedication from a human (on the level of daily focus across weeks) is what the AI can deliver on demand.

Now that we have the basic creative output, we can choose among Claude’s suggestions and tighten them up.

I don’t like this. It feels dangerous. It’s too tempting. Being deeply focused on just one topic for weeks means giving up on a LOT of other things that I don’t want to give up on. But if I compensate by relying on this tool I worry I’ll stop going into deep focus entirely, and just lose the knack for it over time. Why wouldn’t most people do so, if they could? How many humans in the present day can memorize and recite long orations? We don’t need to now that we have written language. Per Plato, have we not leaned close to becoming “hearers of many things”, [appearing] as though they were all-knowing, but to actually be learners of nothing.

Soon we won’t even have to think up our own thoughts. Ten years from now ^[1]the AIs will be doing most of the things we think of as creative-generation and humans will mostly be the Deciders, picking among the options presented to get at the true essence of what we want. A cool premise for a sci fi world, but not a world I would choose.

(this is a mirror of a blog post at DeathIsBad)

1. ^{^}

   Given the standard anti-doom caveats

Definitely not wrong, the petitions almost certainly won't change anything. Change.org is not where one goes to actually change things.

I had a quasi-romantic relationship with a fictional character that lived in my head during my worst year, in college. I could sometimes even "see" him. I knew he wasn't real. It did help me out during the darkest times. Probably woulda been even better to be able to have chat conversations that were run by an AI. And I did outgrow that in a few months, when life got better. 

So, basically, this sounds great and I love this perspective. Thank you.