There's a deep problem with this claim: confirmation bias. You believe LLM writing is bad in certain ways; you see bad writing with certain tics; you update even further toward LLM writing being bad.
Suppose that some LLM writing is good, and doesn't have those recognizable tics. Would it change your belief? Generally no, because you don't identify it as LLM writing. Inversely, suppose some writing with those tics is by humans. Would it change your belief? Again no, because you would assume it was written by an LLM.
Further: if you know in advance that a piece of writing is by an LLM, that's highly likely to influence your opinion of it.
I'm not necessarily claiming that some LLM writing is as good as human, but I claim that if it were true it would take you a painfully long time to realize that, because the only cases that would update you would be ones where you formed an opinion and only then learned the provenance of the writing. And how often does that happen?
I think evidence from blind tests is almost the only useful evidence here, because most people already have strong beliefs one way or the other. Prior to seeing such evidence, I think the more reasonable claim is, 'Don't let LLMs produce crappy writing for you', which really can be simplified into 'Don't put out crappy writing'.
I infer you mean "the claim that LLM writing is a slog to get through." Which yes, I wait for the day that most LLM writing I see is not usually a slog to get through. I hope it comes! It's perfectly possible I see lots of LLM writing that was created by prompting wizards using state-of-the-art $200-a-month models in ways not dreamt of in my philosophy. If so, great. If you can fool me, we both win.
But I see so much LLM writing in the wild and professionally that just sucks. Whether it sucks because of some fundamental property of LLMs (doubtful), or because of path dependencies for the LLMs that are commercially available (getting warmer), or because of bog standard skill issues on the part of the relevant centaur (could be), one clear fact is that the authors don't think there's a problem.
In situations like this, where I see a lot of people doing something that seems like a pretty big mistake, I want to just say "don't do that." Because I think if I say "don't do that, unless you're good at it," well, if the people making the mistake knew they weren't good at it, they'd already not be making the mistake! I'd rather say "don't go cave diving" and the tiny minority of expert, professional cave divers who know and relentlessly apply all the proper cave diving safety rules can smile knowingly and ignore me. Of course, the amateurs can ignore me too. But I am here advising otherwise!
1Thanks for the thoughtful reply!
I infer you mean "the claim that LLM writing is a slog to get through."
I mean centrally the claim 'to most human beings, AI prose is something sus. If you use AI to write something, people will know. Not everyone, but the people paying attention, who aren’t newcomers or distracted or intoxicated.' I should have been clearer about that.
It's specifically that factual claim that I think tends to be strongly self-reinforcing. If you don't have a way to identify false positives and false negatives, there's no way to get evidence that updates you against that claim.
I'm intending to make a broad epistemic claim here that's not specific to LLM detection. One case that really drove this point home for me personally was the great 'do mp3s sound worse?' debate a number of years ago. Many people were utterly confident that they could hear the difference between mp3s and uncompressed files -- and of course they could in some cases, when the mp3s were poor quality! But in the many cases where they couldn't, they assumed they were hearing uncompressed files, and so it didn't change their minds at all.
But I see so much LLM writing in the wild and professionally that just sucks
I absolutely agree that there's a ton of sucky writing out there, including lots of sucky writing that's identifiable as being from LLMs (and probably some sucky writing that isn't identifiable as being from LLMs). Ninety percent of everything is crap.
In situations like this, where I see a lot of people doing something that seems like a pretty big mistake, I want to just say "don't do that."
Fair point! I'm not necessarily disagreeing with the advice, only with the epistemic validity of the factual claim.
That said, separately from the epistemic point: after posting the comment, I got curious about the current state of the evidence on LLM writing detection by humans. I asked a couple of frontier models[1] to do a review of 2025 / 2026 papers using blind testing[2]. As I read it, the upshot is:
- Most readers can't tell the difference better than chance.
- Expert LLM users and those with training specifically in LLM detection do better, although evidence is somewhat mixed. Notably, this was in significant part because they had learned specific tells rather than judging the output as low-quality.
- Subject matter experts do better for some types of writing, less so for others.
- There are simple interventions (eg get the model to play a particular persona) that make the writing much harder to identify as LLM.
Note that most of the data is on 2023/2024-era models, eg ChatGPT-3.5, GPT-4o, Claude 2. I would expect that humans would do worse at detection with current frontier models.
Yeah, it's an interesting question how good human detection is. My guess is that people who are paying attention are getting better at sniffing out AI faster than AI is getting less distinctively scented, but "people who are paying attention" is a heck of a sleight of hand.
Overall, I suppose my main feeling is that I see AI generated stuff all the time in lots of different arenas, and I see other people judging it, and it sort of feels like an Eternal September where some people are freshly excited by some AI use case in their thinking, and don't realize how it comes off (or do, but it hasn't occurred to them that it comes off badly for good reasons as well as straightforward prejudice). There may also be lots of people using AI so skillfully that they don't fall into any of these traps. It's even possible they far outnumber the people who are (I think) bumbling. Perhaps my doubt for this last proposition is a stuck prior. But if so, it is well and truly stuck.
1Absolutely agreed re: people naively using LLMs in the simplest possible way for writing and that coming off very badly. And I think getting such people to consider the issue is well worthwhile!
1Reminds me of the - "No CGI" is actually Invisible CGI - video series from a few years ago. Linked below. At this point, a substantial percentage of writing is at least aided by LLMs so the only LLM writing you notice is just the bad writing. I don't fully buy this but I think it explains a least a portion of the discourse.
1It is possible to get evidence for this claim without blind tests. For example: start interacting more with prose from an LLM you don't interact with often (I recently discovered that I like Kimi K2.5's prose much better than Claude's, for example, so I'm interacting with it more). Track your ability to distinguish that LLM's outputs (and your subjective taste/distaste for those patterns) over time. If you start to dislike tics that you didn't notice before, that's reasonable evidence that you've come to associate those tics with writing that lacks the sort of interiority described here, or at least with writing that lacks some desirable quality that's hard to specify.
Well, there are plenty of long takes on X which are obviously based on authors' ideas but LLM-generated (even before ones runs them through a detector) and still get pretty popular, audience not smelling an LLM. Do you count that as good or bad writing? I honestly don't enjoy reading them for some reason even when I agree the underlying ideas make sense, but on the other hand, these authors reached a wider audience than they would presumably have without an LLM
Do you count that as good or bad writing?
Not sure if you meant to ask Justis that? My own standards for writing are (as I understand them) mostly independent of information about the author (there are some exceptions like autobiographies, or knowing the author has overcome some incredible hurdle to be able to write it, but they only apply in a small percentage of cases).
I also, separately, don't think popularity on X is a very useful proxy for writing quality.
I think the best objection to LLM writing goes "If I wanted to know what an LLM thought I would ask one." Everyone with $20/month to spare already has all the LLM commentary they want and there is no need to show your audience more.
Unless they're bad at prompting or just aren't going to go run the prompt. I'm considering writing an article where the actual point are the prompts, and if a human reads the prompts and takes only the prompts away then that's actually just fine, but where LLM output might clarify as a cheat sheet if someone finds the prompts inscrutable. I'm not sure how this will be received, I suspect the allergy to LLM writing is so strong that even doing that will annoy people and get strong downvotes. Maybe I'll only include the prompts. But I think my user preferences prompt produces better output than other people will get if they rerun the prompts, and I'm not going to share my user preferences publicly.
This does seem valuable.
I'd put the responses in collapsible boxes.
What about sharing the parts of your user prompt that you think are most important, removing information where you want? I do suspect that a good user prompt is very helpful in getting non-BS answers from LLMs. I have to change mine when model versions change their level of sycophancy.
I guess it will be fine if you directly describe what you're doing and you make it easy to only read the prompts / skim the LLM outputs. Because few people share their LLM prompts when posting LLM writing, people are not fed up by the exercise. If anything it sounds like a "small fun experiment".
Everyone with $20/month to spare already has all the LLM commentary they want
Yes, but people update slowly. Learning to use an LLM at every opportunity is a skill that may take a few years to learn, just like learning to use Google at every opportunity did.
When someone shares a clickbait story on Facebook, I ask an LLM to fact-check, and then I reply with a short summary. Some people seem to appreciate that. But any of them could have done the same; they are just not aware of that at the moment.
I'm quickly finding lots of caveats.
On the one hand I don't let Claude fully write for me. Even if I let it write a draft based on an outline, I so thoroughly rewrite that draft that it's not really Claude's anymore. It's more like a bunch of babble in the direction of what I want that I can use to prune into what I want to say.
But on the other I find all kinds of docs where basically nothing is lost by letting Claude lead the writing.
Tech specs? Yep, I just let Claude own the words. All I care about are the decisions, and there's little value in exactly how those decisions are expressed in words so long as the decisions are communicated clearly.
Research paper? Again, mostly can just let Claude own it. The words are just a means to convey some results. No one is reading this for the prose.
Claude skills? Also just let Claude write them. Yes I might make some small tweaks in the name of token efficiency, but it knows how to write for itself better than I can.
But an essay on my blog? A chapter in my book? Well, those places are where I express my ideas, and Claude can't understand them better than I can unless I first clearly express them myself, at which point all Claude can offer is additional ways to explain the same ideas (which is sometimes helpful in editing!).
So in the end I find it really depends on the purpose of the writing. If it's about the words and ideas, then I agree, don't let LLMs write for you. But if it's not so much about the words as about what the words are used to convey, then the words aren't adding a lot of value by being written by you and you can safely let the LLM write them, just as I like the LLMs write code.
I agree with this directionally, but would add a caveat to your research paper caveat. If it's a research paper of the type 'I ran experiment X and found Y' then yes, I suppose the words don't matter so much beyond conveying the result - although I've found Claude to have a tendency to overclaim and overinterpret results that needs to be carefully kept in check, to the extent that I often think "man it'd have been easier to write this myself!"
But 'I ran experiment X and found Y' is not what every research paper is! For many research papers, I definitely care about the ideas of the author and how they convey them, so I'd rather they stay mostly LLM-free.
or any number of other ineffable signs subtler than an em dash.
I just started using em dashes (at the recommendation of my older brother) in my writing circa 2022 before I even knew about LLMs. I like using them—I'm not giving them up!
1
1that didn't trigger any allergic reaction for me ... but following would:
I just started using em dashes—at the recommendation of my older brother—in my writing circa 2022 before I even knew about LLMs. I like using them and I'm not giving them up!
It's not your fault or ours, just an unfortunate fact: we will by default assume your stuff was written by an LLM and pay it less attention, for the reasons given above.
Unless you include a statement in each piece that you're using them.
I also used a lot of em-dashes and loved them. There are better hills to die on.
Wait—really? I have a long track record of using em dashes and, I think, a pretty distinctive writing style. I've had a dash binding in my .emacs since 2015. Can people really not tell? If you're going to reject something as presumptive slop because of a em dash, isn't that confessing that your discernment is so low that there's no reason for you to avoid the slop?
It's a weak signal. I've certainly never suspected your writing of being LLM, because the rest of it screams human and high quality. What Even Is this Timeline? just nails it, and no LLM to would produce that without enough prompting to make it worth reading. I guess I'm biased because I think the type of work you're doing, taking stock of the big picture with an understanding of different perspectives, is word for word the most valuable thing right now. I'm just mentioning, not trying to distract you from the issue at hand.
For writers I don't know, I'm afraid yeah em-dashes are a signal to pay it less attention. I'm not the LLM-discernment pro that the mods here are. I can tell when it's obvious but not when it's not. I don't spend a lot of time learning that discernment. And I'd assumed that everyone with a clue made the same compromise I did and ditched them when LLMs made them their signature move.
But I didn't say I stop reading, like Justis did. I'm not as hardline. I skim LLM-written posts for good ideas, because I think insights on alignment (and other things) can and do come out of left field, and writing ability is weakly or even anticorrelated with creativity (while correlated strongly with analytical ability).
I think it's an unfortunately situation because LLM writing is merely correlated with lack of quality; I think the case is overstated, perhaps because we mostly hate and fear AI for good reasons, and our feelings cause some motivated reasoning. But it is what it is. For now. I hope it will be different when LLMs have better metacognitive skills and instead of skipping LLM writing we can just say "make sure you tell your LLM to think this back and forth carefully and write succinctly"
I think that might happen soon because commercial use of LLMs will reward accuracy more than sycophancy. I hope to be alive to enjoy that glorious few months when LLMs can think for us better than we can, and still want to.
But for now, yeah, there's already too much to read, so em-dashes are a sign to start skimming.
1If you're going to reject something as presumptive slop because of a em dash, isn't that confessing that your discernment is so low that there's no reason for you to avoid the slop?
Unfortunately no, I don't think so, because people who want to avoid wasting their time on LLM writing are likely to be quite sensitive to signals of LLM writing and potentially very quick to nope out. Generally it is (or at least feels) less costly to miss out on a random blog post than it is to ingest meaningless writing. So if there's an early sign in your writing, someone who cares probably won't stick around and read through to the end to evaluate based on the entirety of the post. (Unless they see other signals that it's a high-quality post, like if other people are recommending it - in which case they will probably read it even if they think LLMs were involved in the writing.)
I happen to have known of your writing before but honestly this comment gives me a bit of pause lol. I would have considered an em-dash in online text "not getting the memo" as often other "AI-ish signs" are not as clear and blatant. Random people who see your writing won't know you (generic you) used it back in 2015.
And yes I find there is way too much spam/scam/AI generated text online nowadays to not skim over a lot.
There's one group of people to whom your advice probably doesn't apply, I mean, the vast majority of people: most of us who write scientific papers are non-native, often even Bad English speakers (me included). Many non-native speakers use LLMs extensively, for good reasons. In fact, this is a staggering change for the better in their lives! Imagine: much faster writing, almost as if you wrote in your mother tongue; no need for "native speaker editing" anymore: you don't have to beg your native speaker colleague to do it for free because you don't have funding for experts. You personally may prefer reading broken English, but there are problems with that: A) journals won't accept it, and B) broken English makes the author sound a bit mentally challenged. LLM-generated Perfect Officialese English makes one sound like a passably intelligent though humourless person, which is, unfortunately, better.
Almost all PhD position application letters from non-native speakers are written in LLM Officialese nowadays. Since it's so common - basically everybody - we don't judge them by that aspect, it doesn't convey any information. Of course, if a letter comes well-written in a human style, it stands out favourably.
Yes, I agree. There is a growing premium on being a native english writer these days to prompt/correct the LLM to sound human. The rest of us stumble in the dark when we use an LLM do get our texts in order.
I typically argue for a slightly more modest policy: It's fine to use LLMs for writing, but you should disclose it. In practice this isn't too different from a "don't use LLMs" policy (since a majority of people will decline to read it if you do disclose) but it's much harder to argue with!
1I go on LinkedIn once in a blue moon. Now though, whenever I am on the LinkedIn homepage all I see is AI writing. It's every post that has a "It's not this arbitrary thing I made up. It's that arbitrary but slightly more connected thing I also made up."
I think that most people don't care about their LinkedIn posts and just want to get likes.
And I don't care about LinkedIn. But it's an affront to the senses when I have to sign in to update my resume. It makes me feel like "wow what a garbage place to be there's so much content here and none of it has any value."
When compared to coding which benefits from consistent structure? (not sure this is what I mean), it's the varied prose of humans that is still yet to be replicated by LLMs that's so endearing. It's not even just the pointer that this person spent little to no time on this writing.
And it's interesting that this hasn't been solved yet and the recent attempts to do so by even OpenAI are so minuscule (ex. telling it to calm down on the em dashes in the settings).
So for the foreseeable future I think I'll still actively dislike AI writing when I see it in the wild.
human writing is evidence of human thinking. If you try writing something you don’t understand well, it becomes immediately apparent; you end up writing a mess, and it stays a mess until you sort out the underlying idea.
Can you elaborate more on this. It feels like quite the opposite to me - the more I've thought about something, the messier it comes out. The harder it is to unknot the spider-web of thoughts into a linear rhetorical structure which is readily comprehensible to a virgin reader. Particularly topics I have a tendency to 'geek out' on. Does this mean that I don't truly understand them, or that they are lacking a unifying underlying idea? Am I perhaps confusing passion and knowledge for understanding?
Or is it only evidence of thinking about the writing - the words on the page/screen the reader is looking at right now? And one can have a personal understanding of something which is clear in their own head (or perhaps even readily conveyed to others with similar domain knowledge - like that XKCD comic), but not readily translatable to the page?
I think there are two separate claims lurking here:
- If you don't have a clear idea, your expression of it will be a mess
- If you do have a clear idea, your expression of it will be clear
I think the first is basically true, while the second isn't necessarily true because writing can be hard. There are lots of ways for a clear idea to not make it into clear writing, like inferential distance as you mention.
The main thing to me is, if you start writing and it's just not working, one hypothesis is that your thinking has still not really crystallized. And if you go straight to an LLM to "clarify this" you accidentally tend to throw out that hypothesis.
1What's your explanation for how LLMs can create non messy writing without conveying a clear point, and in your view do humans ever do it? I'm finding myself simultaneously agreeing with your first claim and also thinking "but English class", and I'm not sure how to square these conflicting ideas.
English class always seemed to me to be about saying absolutely nothing in flowery language. Almost like a politician speech, where the point is to talk in such a way that the point is to make the words sound so nice that you don't notice that the content is missing. Only I don't think it's intentional, just a byproduct on focusing on the language without having anything interesting to say.
And if you go straight to an LLM to "clarify this" you accidentally tend to throw out that hypothesis.
I'm not sure how to ask this question - but can writing cultivate understanding, even in the absence of new data about the theme or topic? And when I, or anyone, goes straight to an LLM to clarify an undercooked idea, or theory, or network of thoughts, they are not only outsourcing the work to express it verbally, but also are missing out on an opportunity to think and understand? As per the cliche "writing is thinking".
You have no idea how many times I've tried to redraft this question, all while resisting the urge to get an LLM to rephrase it for public consumption.
1If I'm understanding your question correctly (it seems clearly written, but the answer seems so obvious I'm doubting myself)... yes absolutely and it's the standard tool for doing so! That's the basis of personal journaling, or tech blogging, or many other forms of writing.
Ah, now I know how to phrase my question, it's really two questions:
1. What distinguishes understanding from knowledge (or even passion about a topic)?
2. How can I write for the express purpose of understanding better? Presumably, not all manners of writing and jouralling are equally conducive to promoting understanding. And as such it's not enough to write, or not-out-source to an LLM, there's a particular method or way of thinking and composition of text which will improve the results.
On the first point - there's plenty of things I can geek out about and wax lyrical - but it comes out as a mess and impossible to compose into a linear structure suitable for a virgin reader. Does this mean I don't understand?
On the second point - I haven't seen or enjoyed the benefits that others get from journalling or other forms of writing in understanding. I gain a lot more from dialogue (see how I finally figured out what my question was above), and FAFO: just doing the thing. I presume this means I'm doing writing wrong.
Re: understanding,
I'd loosely describe the difference between knowledge and understanding as the difference between being able to say what something is vs being able to describe why it is, or how it is, which often comes through being able to describe the thing in different ways. See the concept of "you don't really understand something until you can explain it to a child (or lay person, I'd say)."
I know what a GPU is - I don't understand how it works on a physical level.
Passion seems orthogonal although it csn drive knowledge and understanding.
About writing, well, our brains are all different - no technique will work equally well for everyone. Dialogue is a great way to generate understanding. And it has precedence as a writing technique too - have you tried writing fictional dialogues to hash out your ideas?
Paul Graham has some good pieces on this:
I personally think you're confusing knowledge with understanding. If you know a ton about a topic but can't explain it clearly to a novice, you have a lot of knowledge of the details but not something we might call understanding, or knowledge of how it all fits together and why someone might/should care about any of it.
I agree that writing is often crucial for me to resolve my knowledge and thinking about a topic into understanding of that topic.
If you know a ton about a topic but can't explain it clearly to a novice, you have a lot of knowledge of the details but not something we might call understanding, or knowledge of how it all fits together and why someone might/should care about any of it.
How do you know if the topic is just unrealistic to get a novice up to speed, or if you're not actually understanding it? Are there tell-tale signs?
What is understanding and what obvious or immediately apparent traits does a mind that has understanding about a topic differ from one that has maintained a large body of knowledge but not "understanding"?
These are good questions, and I don't have good answers.
Of course my use of "understanding" isn't that common, let alone universal; but I think it's an important concept to have, even if it's vague.
Easily and clearly explaining to a novice without resorting to others' explanations or cached thoughts is the best tell-tale sign I know of.
I do say a little more about this definition of understanding in my Sapience, understanding, and "AGI" . But not much.
1If the guy asks to “help find citations” and there are no actual good ones
When it comes for getting LLMs to help with citations I usually put the LLM in Deep Research mode and ask: "Can you do background research about the claims in this paragraph?" This does manage to reveal when claims in the paragraph are out of touch with the actual research in the field and provides good citations which you wouldn't get if you ask directly for citations.
If you have what feels like an original idea and there's a scientific field touching it while you are a lay person, running these kinds of deep research queries is probably a good idea even if you write all the actual words yourself. There are plenty of blog posts where a person has their own idea that has shown to have flaws in actual research or the actual research has already found great terminology for it.
I think the arguments about usefulness of unaided writing for your thinking and for your productivity should be clearly separated and treated separately. Consider physical activity: we surely agree that physical activity is essential for health. But that doesn't imply that it is a good idea to go to work as loader. It is quite likely that soon manual writing will be treated rather as a "mental gym".
I agree with the core claim, but I'd draw the line at LLMs writing for you, not at LLMs helping you revise.
The bad version is: "take my half-formed idea and make it sound smart".
The good version is: "tell me where this draft is unclear, flat, or overclaimed, and I'll rewrite it myself."
That distinction ended up mattering so much to me that I built Draftly around critique instead of generation: https://joindraftly.com
For me the value is in surfacing the weak spot, not in having the model supply the prose.
Disclaimer
I’m not a native English user, so some of the text I wrote could appear broken or unclear. Apologies for any inconvenience in reading.
AI use is limited to grammar check only, 1 exception is noted in footnotes [2]. The tool used is Claude and Writing Tool[1].
Google Translate is used for single-word and short phrase translation.
Most of your take is established on the fact that AI generated text is not good enough and is easily distinguishable from human written one, and I agree on that, but I have no doubt that someday it will improve to a level where you cannot tell it apart from human written text.
I think we should focus more on whether people should do it even if nobody is going to catch them, it’s about writer ethics, I wouldn’t mind the AI use if an author clearly marked which part is generated or modified by AI and how AI is used. But not being transparent about this is irresponsible to readers.
If you use AI to write something, people will know. Not everyone, but the people paying attention, who aren’t newcomers or distracted or intoxicated. And most of those people will judge you.
The bias problem a lot of people have mentioned. It’s hard to say if your presumptions are correct in the first place, so yes, AI-written text has identifiable patterns, but it is hard to confidently tell that not a line in the articles you think are human-written is actually written by machine. This presumption does feel too absolute. (I saw your reply under another comment that addresses this issue)
Plus, in which context are you talking about writing? Science, literature, news, messaging… the audience expectations of these fields vary a lot. AI-generated text is accepted or rejected for various reasons.
For general long content writing, I think using AI to directly generate text on a topic with vague descriptions can be considered plagiarism in some way, it is unearned authorship[2] as the thought does not come from the human author at all. They should not even be treated as original work.
My opinion is that, in this era where machines are as intelligent as humans in some domains, content consumers should have the right to know what is actually behind the content they are spending time on.
many people use llms to write garbage. But this was the case even before llms, e.g. article farms. I would say that llms have raised the bar in such cases.
I let the llm write for me. I prompt it for the concepts to write about, read it, tell it to rephrase something I don't like. The initial prompt (before the article key ideas) includes information about me, such that the output is closer to what I would express things with. The output is blatantly still LLM sounding.
Could I just put the prompt in my blog and let others ask their own llms? That wouldn't convey what I want. There is still value in the right prompt as the possible questions are uncountable. There is value in tuning the output.
To retain some human touch I prefer to manually write the footnotes in places where I don't like the phrasing of the llm that much but changing it would break the flow.
English standardized international language in the last half century, through the internet. LLMs are standardizing the English form. Yes, its kinda Orwellian, but compared to news papers and televisions, they are not worse, but better. You can say that old media does a worse job at plasmating human minds with repetitive and often times subliminal messages, but there is variance in the media sources. LLMs definitely provide much more variance in their outputs, but less in their sources.
Speed reading is as important as it has ever been. LLMs generate content with proper bolding, pacing, keywords better than your average article. If you think llm content feels empty, just speed read through it, tune your attention to the hints the content gives you.
Content note: nothing in this piece is a prank or jumpscare where I smirkingly reveal you've been reading AI prose all along.
It’s easy to forget this in roarin’ 2026, but homo sapiens are the original vibers. Long before we adapt our behaviors or formal heuristics, human beings can sniff out something sus. And to most human beings, AI prose is something sus.
If you use AI to write something, people will know. Not everyone, but the people paying attention, who aren’t newcomers or distracted or intoxicated. And most of those people will judge you.
The Reasons
People may just be squicked out by AI, or lossily compress AI with crypto and assume you’re a “tech bro,” or think only uncreative idiots use AI at all. These are bad objections, and I don’t endorse them. But when I catch a whiff of LLM smell, I stop reading. I stop reading much faster than if I saw typos, or broken English, or disliked ideology. There are two reasons.
First, human writing is evidence of human thinking. If you try writing something you don’t understand well, it becomes immediately apparent; you end up writing a mess, and it stays a mess until you sort out the underlying idea. So when I read clear prose, I assume that I’m reading a refined thought. LLM prose violently breaks this correlation. If some guy tells Claude to “help put this idea he has into words” then Claude will write clear prose even if the idea is vague and stupid. If the guy asks to “help find citations” and there are no actual good ones, Claude will find random D-tier writeups and link to them authoritatively. Worst of all, if the guy asks Claude to “poke holes in my argument” when the argument is sufficiently muddy, Claude will just kind of make up random “issues” that the guy will hedge against (or, let’s be real, have Claude hedge against). So you end up with a writeup which cites sources, has plenty of caveats, and… has no actual core of considered thought. If you read enough of these, then you start alt-tabbing away real fast when you see structured lists with bold headers, or weird clipped parenthetical asides, or splashy contrastive disclaimers every 2-3 sentences, or any number of other ineffable signs subtler than an em dash.
Is it possible that a 50% AI-generated hunk of text contains a pearl of careful thinking, that the poor human author simply didn’t have the time or technical skill to express? I suppose. But it ain’t worth checking.
Second, and closely related, AI prose is a slog. There’s way too much framing, there are too many lists and each list has a few items that serve no purpose, the bold and italics feel desperate, and it’s just all so same-y. In your own conversation with an AI that you can fully steer, you can sometimes break out of this feeling for a little bit. But reading the output of someone else’s AI conversation is rarely any fun.
In short, if someone reads writing “by you” and it seems LLM-y, they will think both that:
If you want them there, they are not going to stick around. In fact, the more you want a reader, the more likely they are to be turned off by this stuff. Even if they’re the biggest AI fan in the world.
Luddite! Moralizer!
Fine. I admit it. Just this week, I too experienced Temptation.
You may know me as an editor. In this capacity, I was revising an academic paper’s abstract in response to reviewer comments. But I had several papers to work on in the same project, and the owner of that project actively encouraged me to use AI to move fast enough to meet deadlines.[1]
So I gave Claude the paper and the reviewer comments, and asked it to come up with a new abstract that would satisfy the reviewers. The result looked good.
“It’s just an abstract,” I whispered to myself, face lit eerily in my laptop screen’s blue light. “Summary. Synthesis.” I rocked back and forth. “I could… just…”
But no. Claude’s abstract was a useful reminder of which paper this was, and Claude helpfully catalogued what the reviewer requests were. Still, I rewrote the abstract myself, from scratch. In so doing, I noticed a lot of things I hadn’t seen, when I was just skimming the AI output. Stuff it included that it didn’t really need to. Stuff it emphasized that wasn’t actually that important.
Did I run my abstract by Claude in turn? Yes! It had two nitpicks, one of which I agreed with, and fixed in my own words. Use these tools. You should totally ask Claude to find you sources for a claim, but then you should check those sources like you would check the sources of an eager day one intern, and expect to throw most (or all) of them away. You should totally ask Claude to fact check, but expect it to miss some factual errors and unhelpfully nitpick others. You can even ask Claude to “help clarify your thinking.” But if you’re really just clarifying it, then you won’t use its text. Because once your thinking’s clear, you can write the text yourself, and you should.
To be clear, editing I do as part of the LessWrong Feedback Service uses my own human judgment, and I don't use LLMs to make edits.