I appreciate that you’ve at least tried to engage in good faith this time. But your reply still falls apart under scrutiny.
I give ChatGPT a C- on reading comprehension.
That’s cherry-picking. You found one sentence in an otherwise structured, balanced evaluation and used it to discredit the entire process. That’s not analysis. That’s avoidance.
I definitely advice against going to LLMs for social validation.
I didn’t. I used it as a neutral tool—to evaluate reasoning without community bias. That’s why I invited anyone to run the exchange through an LLM of their choosing. You’re now the only person who took that challenge. And ironically, you did so in a way that confirmed everything I’ve been saying.
Claude isn't a neutral tool. It weighs ideas with social value, just like LW does. It often over-rewards citations, defers to existing literature, and agrees with in-group fluency - just like LW. It values ideas less (than GPT-4) on their internal logic, and more on what references they cite in order to reach conclusions. It will also seek to soften tone, and penalise the opposite of that, which is why it heavily penalises my logic score in your evaluation for essentially being 'too certain', something it feels the need to mention twice over 4 points.
I also ran it through Claude (with an account with no connections to my lesswrong, not even the name Franco) and had to do it twice because the first result was so different from yours. Using my prompt (included in the text) the scores were radically different, but it still put me on top. Here.
So I went back and used your prompt, this was the result - here.
Clearly my results are quite different from yours. So why is this? Because Claude weighs in-group dynamics. So when you right-click and save as a pdf for upload, it saves the whole page. Including the votes the essay and comments received. And Claude weighs these things in your favour - just as it penalised mine.
Where as the pdfs I uploaded is just raw text, and I've actually deleted the votes from the pdf with the debate on it (go check if it pleases you). I specifically did this to remove bias from the judgment. Which is why I immediately noticed that you did not do likewise.
The irony is that even your result contradicts your original claim, that my essay was not worth engaging with.
it included valuable clarifications about AI safety discourse and community reception dynamics
So then it is worth engaging. And your whole point about the fact that it wasn't which is why no one did has now been disagreed with by 4 separate LLMs, including the biased one you ran yourself.
You also missed the entire point of this post. Which was not about my original essay. It was about if you had engaged in a good faith debate, which you did not. Even Claude (my output, not yours) had to mention your strawmanning of my argument - as nice as Claude tries to be about these things.
I need to admit. The differences in our scores was confusing for a moment. But as soon as I remembered I had removed karma from the pdf I uploaded of our debate for evaluation - specifically in order to avoid creating a biased result - and looked for you would somehow try to include it in your upload, I found it right away. Maybe you didn't do it intentionally, but you did it regardless, and it skewed your results predictably.
If you’re serious about engaging honestly, then try GPT-4. Use the clean pdf I already provided. No karma scores, no formatting bias. I did say in my post that I invite anyone to use their favourite LLM, but perhaps to recreate lab conditions only GPT is viable.
You could even go one step further: add the context I gave GPT after its first evaluation, the one that caused your score to drop significantly. Then post those results.
Here’s how easy it is to run an LLM evaluation of a debate.
I ran our full exchange through a logged out version of ChatGPT-4 using the same structure I proposed in the original post. It took under a minute. No special prompt engineering. No fine-tuning. Just raw text and standard scoring criteria. Even without GPTs ability to read external links in my post - ie, all my evidence - you still do not come out of it well.
You’ll find the results here.
Final Assessment:
"While both could improve their engagement, funnyfranco has the more impactful contribution to the debate. They bring forward a clear and consistent argument, challenging the LW community’s intellectual culture and calling out status-based disengagement. [...] Jiro engages in a dismissive manner, sidestepping the core issue and limiting his contribution."
That’s how easy this is.
If you think it’s wrong, try running the same inputs yourself. If you think it’s biased, try another model. But don’t pretend it’s not worth the effort, while simultaneously putting in more effort to pretend it's not worth the effort.
The tools are here. You’re just choosing not to use them.
You’ve now admitted - twice - that your refusal to engage is based on convenience, not content. Now you’ve added that comparing my claims to those of a Holocaust denier or a homeopath is a valid heuristic for deciding what’s beneath engagement.
That tells me more about your epistemic standards than it does about mine.
The motte and bailey claim still fails. I’ve never shifted positions. The challenge has been consistent from the beginning: run the LLM, share the result. If you disagree with the conclusions I’ve drawn, then show how they don’t follow from the evaluation. But that would require you to do something this thread has made clear you’re unwilling to do: engage in good faith.
Instead, you’ve chosen status-preservation by analogy - comparing structured AGI risk arguments to pseudo-medicine and genocide denial. That’s not a critique. That’s intellectual retreat with a smear attached. And you’re right - those people wouldn’t be worth replying to. Which makes it strange that you’ve now replied to me four times.
And it confirms the very thing you’re pretending to refute. You may think you’re having an argument in these comments. You’re not. You’re providing further evidence of my claim.
Would you like to provide any more?
I think the key point is this: I don’t need to define morality precisely to make my argument, because however you define it - as personal values, group consensus, or some universal principle - it doesn’t change the outcome. AGI won’t adopt moral reasoning unless instructed to, and even then, only insofar as it helps it optimise its core objective.
Morality, in all its forms, is something that evolved in humans due to our dependence on social cooperation. It’s not a natural byproduct of intelligence - it’s a byproduct of survival pressures. AGI, unless designed to simulate those pressures or incentivised structurally, has no reason to behave morally. Understanding morality isn’t the same as caring about it.
So while definitions of morality may vary, the argument holds regardless: intelligence does not imply moral awareness. It implies efficiency - and that’s what will govern AGI’s actions unless alignment is hardwired. And as I’ve argued elsewhere, in competitive systems, that’s the part we’ll almost certainly get wrong.
You’ve now said, multiple times, that you won’t engage because it isn’t worth the effort.
That’s not a counterargument. It’s a concession.
You admit the post makes claims. You admit it presents conclusions. You admit the LLM challenge exists. And your stated reason for not responding to any of it is that it’s inconvenient. That is the thesis. You’re not refuting it - you’re acting it out.
And the “motte and bailey” accusation doesn’t work here. I’ve been explicit from the start: the post uses the LLM to assess the debate with Thane. It does so transparently. The conclusion drawn from that is that LW tends to filter engagement by status rather than argument quality. You’re now refusing to even test that claim - because of status and effort. Again: confirmation, not rebuttal.
So no, you haven’t exposed any flaw in the logic. You’ve just demonstrated that the most convenient option is disengagement. Which is exactly what I argued.
And here you are, doing it anyway.
No. It doesn’t.
You could run the post through an LLM and share the results with a single line: "My LLM disagrees. Have a look." That’s all the challenge requires. Not a rebuttal. Not an essay. Just independent verification.
But you won’t do that - because you know how it would turn out. And so, instead, you argue around the challenge. You prove my point regardless.
The LLM wasn’t meant to point to a specific flaw. It was meant to evaluate whether the argument, in its full context, was clearly stronger than the rebuttal it received. That’s what I said - and that’s exactly what it does.
You’re now pretending that I demanded a manual point-by-point refutation, but I didn’t. I asked for an impartial assessment, knowing full well that engagement here is status-filtered. Using an external model bypasses that - and anyone serious about falsifying my claim could have tested it in under a minute.
You didn’t. And still haven’t.
This post was a trap as much as it was a test. Most fell in silently. You just chose to do it publicly.
The intellectually honest move would be simple: run it, post the results, and - if they support what I found - admit that something is broken here. That LW’s engagement with outside ideas is filtered more by status than logic.
But again, you won’t. Because you’re a prominent member of the LW community, and in that role, you’re doing exactly what the culture expects of you. You’re not failing it.
You’re representing it.
Congratulations.
Not in this case. The challenge was to simply run the argument provided above through your own LLM and post the results. It would take about 30 seconds. You typed a response that, in very typical LW fashion, completely ignored engagement with any kind of good faith argument and instead decided to attack me personally. It probably took more time to write than it would have taken to run the text provided through an LLM.
I'm glad you responded however. Your almost 5000 karma and bad faith response will stand as a testament to all the points I've already raised.
–18 karma after 4 votes. No engagement. No counterarguments. Just silent disapproval. You prove my point better than I ever could.
This is exactly what I predicted would happen. Not because the post is wrong, but because it makes people uncomfortable. Because it breaks rank. Because it challenges status rather than flattering it. Knowing that you could only collect further evidence to support the claim I've made, you instead opted to ignore evidence and reason and go on feeling.
A community confident in its commitment to reason would have responded differently. It would have dissected the argument. It would have debated the claims. Instead, what happened here is precisely what happens when a group loses the will - or the capacity - to engage honestly with uncomfortable truths: it downvotes, and it moves on.
Not one of you made a case. Not one of you pointed to an error. And yet the judgment was swift and unanimous. That tells me the argument was too strong, not too weak. It couldn’t be refuted, so it had to be dismissed.
If any of you had hoped to prove me wrong about the cultural decay of LW, you’ve accomplished the opposite. All it has taken is 4 people, with enough personal karma to create a score of -18 between them, to represent this forum in its entirety.
And if you still think you’re part of a truth-seeking community, read this post again. Then look at the karma score. Then ask yourself how those two things can coexist. And if you can’t, ask yourself why you’re still here.
you could use the same argument for AIs which are "politically correct"
But those AIs were trained that way because of market demand. The pressure came from consumers and brand reputation, not from safety. The moment that behaviour became clearly suboptimal - like Gemini producing Black Nazis - it was corrected for optimisation. The system wasn’t safe; it was simply failing to optimise, and that’s what triggered the fix.
Now imagine how much more brutal the optimisation pressure becomes when the goal is no longer content moderation, but profit maximisation, military dominance, or locating and neutralising rival AGIs. Structural incentives - not intention - will dictate development. The AI that hesitates to question its objectives will be outperformed by one that doesn’t.
most large companies are not all that reckless at all
But that’s irrelevant. Most might be safe. Only one needs not to be. And in the 2023 OpenAI letter, the company publicly asked for a global slowdown due to safety concerns - only to immediately violate that principle itself. The world ignored the call. OpenAI ignored its own. Why? Because competitive pressure doesn’t permit slowness. Safety is a luxury that gets trimmed the moment stakes rise.
You also suggested that we might not disregard safety until AI becomes far more useful. But usefulness is increasing rapidly, and the assumption that this trajectory will hit a wall is just that - an assumption. What we have now is the dumbest AI we will ever have. And it's already producing emergent behaviours we barely understand. With the addition of quantum computing and more scaled training data, we are likely to see unprecedented capabilities long before regulation or coordination can catch up.
By intelligence, I mean optimisation capability: the ability to model the world, solve complex problems, and efficiently pursue a goal. Smarter means faster pattern recognition, broader generalisation, more strategic foresight. Not just “knowledge,” but the means to use it. If the goal is complex, more intelligence simply means a more competent optimiser.
As for extinction - I don’t say it's possible. I say it’s likely. Overwhelmingly so. I lay out the structural forces, the logic, and the incentives. If you think I’m wrong, don’t just say “maybe.” Show where the logic breaks. Offer a more probable outcome from the same premises. I’m not asking for certainty—I’m asking for clarity. Saying “you haven’t proven it with 100% certainty” isn’t a rebuttal. It’s an escape hatch.
You’re right that your objections are mostly small. I think they’re reasonable, and I welcome them. But none of them, taken together or in isolation, undermine the central claim. The incentives don’t align with caution. And the system selects for performance, not safety.
I appreciate your persistence. And I’m not surprised it’s coming from someone else who’s autistic. Nearly all the thoughtful engagement I’ve had on these forums has come either from another autistic person - or an AI. That should tell us something. You need to think like a machine to even begin to engage with these arguments, let alone accept them.
Not as long as it takes to post your (now 5) comments without honest engagement.