A lot of digital ink has been spilled, and if I were a random commenter I wouldn't think it that valuable to dig into my object level reasoning. But, since I'm the one making the final calls here it seemed important to lay out how I think about the broader patterns in Said's behavior.

I'll start by clarifying my own take on the "what's up with Said and asking for examples?"

I think it is (all else being equal) basically always fine to ask for examples. I think most posts could be improved by having them, I agree that the process of thinking about concrete examples is useful for sanity checking that your idea is real at all. And there is something good and rationalistly wholesome about not seeing it as an attack, but just as "hey, this is a useful thing to consider" (whether or not Said is consistent about this interpretation)

My take on "what the problem here is" is not the part where Said asks for examples, but that when Said shows up in a particular kind of thread, I have a pretty high expectation that there will be a resulting long conversation that won't actually clarify anything important.

The "particular kind of thread" is a cluster of things surrounding introspection, interpersonal-interaction, modeling other people's inner states, and/or interpretative labor. Said is highly skeptical of claims about things in this cluster, and the couple of times I've seen someone actually pursue a conversation with him through to completion the results have rarely seemed illuminating or satisfying. (It seems like Said's experience of things in this space is genuinely different from most people I know. He'll ask for examples, I anticipate giving examples, the examples will be subtle and not obviously real to him. I don't mind that he doesn't believe the examples, but then he'll ask a bunch of followup questions with increasing skepticism/vague-undertone-of-insultingness that is both infuriating and pointless)

I think Said sees himself in often pointing out "the emperor has no clothes", but, I think he just doesn't actually have good taste in what clothes look like in a number of domains that I think are essential for improving the art of rationality. 

I do actually get some value from his first couple comments in a thread – they do serve a useful reminder to me to step outside my current frame, see what what hidden assumptions I'm making, etc. I feel fine doing this because I feel comfortable just ignoring him after he's said those initial things, when a normal/common social script would consider that somewhat rude. But this requires a significant amount of backbone. Backbone is great, more people should build it, but I don’t think it’s super correlated with people who are otherwise intellectually generative. And meanwhile there’s still something missing-stair-y about Said, where he’s phrasing his questions in ways that are just under the radar of feeling unreasonable, until you find yourself knee deep in a long annoying comment tree, so “it’s time to use some backbone” isn’t even obvious.

I do sometimes think he successfully points out "emperor has no clothes". Or, more commonly/accurately, "the undergrad has no thesis." In some cases his socratic questioning seems like an actually-appropriate relationship between an adjunct professor, and an undergrad who shows up to his philosophy class writing an impassioned manifesto that doesn't actually make sense and is riddled with philosophical holes. I don't super mind when Said plays this role, but often in my experience Said is making these comments about people I respect a lot more, who've put hundreds/thousands of hours into studying how to teach rationality (which absolutely requires being able to model people's minds, what mistakes they're likely to be making, what thought processes tend to lead to significant breakthroughs)

Said takes pride in only posting ~1 post a year or so that actually passes his bar for correct and useful. I think this is massively missing the point of how intellectual progress works. I've talked to many people who seem to reliably turn out philosophically competent advances, and a very common thread is that their early stage idea-formation is fragile, they're not always able to rigorously explain it right away. It eventually stands up to scrutiny but it wouldn't be at all helpful to subject their early idea formation to Said's questioning.

Said is holding LessWrong to the standard of a final-publication-journal, when the thing I think LessWrong needs to be includes many stages before that, when you see the messy process that actually generated those final ideas.

I do think there are some important unsolved problems here. I quite liked DirectedEvolution's comment here where he notes:

I do agree with Said that LessWrong can benefit from improved formative evaluations. I have written some fairly popular LessWrong reviews, and one of the things I've uncovered is that some of the most memorable and persuasive evidence underpinning key ideas is much weaker and more ambiguous than I thought it was when I originally read the post. At LessWrong, we're fairly familiar as a culture with factors contributing to irreproducibility in science - p-hacking and the like.

And I do agree in many ways with Said's prior comment:

The time for figuring out whether the ideas or claims in a post are even coherent, or falsifiable, or whether readers even agree on what the post is saying, is immediately.

Immediately—before an idea is absorbed into the local culture, before it becomes the foundation of a dozen more posts that build on it as an assumption, before it balloons into a whole “sequence”—when there’s still time to say “oops” with minimal cost, to course-correct, to notice important caveats or important implications, to avoid pitfalls of terminology, or (in some cases) to throw the whole thing out, shrug, and say “ah well, back to the drawing board”.

[...]

It is an accepted truism among usability professionals that any company, org, or development team that only or mostly does summative evaluations, and neglects or disdains formative evaluations, is not serious about usability.

We're now ~4 years into experimenting with the LessWrong Review, it's accomplished some of the goals I had for it but not all of them. Five years ago, before the first Review year was even complete, we told Said:

I, Oli and Ray will build a better evaluative process for this online community, that incentivises powerful criticism. But right now this site is trying to build a place where we can be generative (and evaluative) together in a way that's fun and not aggressive. While we have an incentive toward better ideas (weighted karma and curation), it is far from a finished system. We have to build this part as well as the evaluative before the whole system works, and while we've not reached there you're correct to be worried and want to enforce the standards yourself with low-effort comments (and I don't mean to imply the comments don't often contain implicit within them very good ideas).

Five years later, we've built part of the evaluative system, but I did update after this year's Review that yeah, we need some kind of faster system as well. I’ve found the comments in this discussion helpful for thinking through what needs to happen here. I’ll hopefully write up a top level post about that.

For now, I agree we probably need to directly incentivize good formative critique. But I don't think Said is actually very good at that in many cases. The best critiques of (say) Circling IMO have come from people who actually understood the good things about Circling, got some value from it, and nonetheless said “but, CFAR still massively overinvested in it” or “the people who do tons of circling get better at relating but in a distorted way, where they go off to circling retreats where everyone is into Openness and Connection, and they don’t do the sort of crosstraining you need to actually also be good at working as a professional or being a good roommate.”

I agree that in the domain of “rationality training”, it’s pretty easy to fool yourself. i.e. Schools Proliferating Without Evidence and whatnot. I think there’s a difficulty that lives in the territory of “it actually does take awhile to hone in on the training processes that work best, and navigating that domain is going to look from the outside like futzing around with stuff that isn’t obviously real/important. (I have thoughts on how to do this better, that are outside the scope here)

...

I note that this comment is focused on particular genre of conversation-involving-Said, which isn't necessarily directly relevant to the case at hand. But it seemed like important background for a lot of the discussion and eventual decisionmaking here.