One general class of solution are tools that satisfy an author's goals in an easy fashion, while keeping discussion as visible/transparent as possible.
An idea Ben and I came up with was having an off-topic comment section of a post. Authors get to decide what is "on topic" for a discussion, and there's an easily accessible button that labels a comment "off topic". Off topic comments move to a hidden-by-default section at the bottom of the comments. Clicking it ones unveils it and leaves it unveiled for the reader in question (and it has some kind of visual cue to let you know that you've entered off-topic world).
(child comments that get tagged as off-topic would be removed from their parent comment if it's on-topic, but in the off-topic section they'd include a link back to their original parent for context)
A common problem that bothers me with my own comment section is comments that are... okay... but I don't think they're worth the attention of most readers. Deleting them (with or without hiding them) feels meaner than the comment deserves. Moving them to an offtopic section feels at least a little mean, but more reasonable.
A related idea is "curated" comments that authors and the mod-team can label, which get a highlighted color and move as high in the comment tree as they can (i.e. to the top of the comments list if they're a top level comment, or the top of their parent comment's children)
A common problem that bothers me with my own comment section is comments that are... okay... but I don't think they're worth the attention of most readers. Deleting them (with or without hiding them) feels meaner than the comment deserves. Moving them to an offtopic section feels at least a little mean, but more reasonable.
Maybe you could invert punishments to rewards, and create an "author highlights" section instead of an "off topic" section?
If you're running a blog and want to apply this approach to comment deletion, then instead of framing it as a "reign of terror" where you mass-delete comments from your blog, you would have an "email the author" field below each of your blog posts and a "featured responses" section highlighting the best emails you've gotten on this topic. 'Being accepted to this scientific journal feels like an honor' is connected to 'being rejected from this journal doesn't feel like an attack or affront.'
I thought about that variant. I think I personally prefer having them actively hidden, because a major thing I want it for is attention management. When there are 100+ comments, I think it's a valuable service to split them into "here's what you should definitely look at if you want to be following key intellectual progress from this conversation" and "here's what you should look at if you want to participate in long, sprawling conversations that I think are more about people engaging socially." (I think the latter's important, just not something I want to force everyone to read).
If the comments are just moved to the bottom, it's not at all clear where to stop reading, and if I see a lot of comments I sometimes feel too intimidated to even start.
For easy reference, here is a summary of the claims and concepts included in this post. (But, note that I think reading the full post first makes more sense).
First: thank you for writing this. I, for one, appreciate seeing you lay out your reasoning like this.
Second:
We plan to implement the moderation log Said Achmiz recommended, so that if someone is deleting a lot of comments without trace you can at least go and check, and notice patterns.
I applaud this decision, obviously, and look forward to this feature! Transparency is paramount, and I’m very gratified to see that the LW2 team takes it seriously.
Third, some comments on the rest of it, in a somewhat haphazard manner and in no particular order:
I have, on occasion, over the past few years, read Facebook threads wherein “rationalists” discussed rationalist-relevant topics. These threads have been of the widest variety, but one thing they had in common is that the level of intellectual rigor, impressiveness of thinking, good sense, and quality of ideas was… low. To say that I was unimpressed would be an understatement. A few times, reading Facebook posts or comments has actively lowered my level of respect for this or that prominent figure in the community (obviously I won’t name names).
It would, in my view, be catastrophic, if the quality of discussion on Less Wrong...
I strongly agree about the circling/kensho discussions. Nothing in them looked to me as if anyone was saying it's not OK to talk about fuzzy system-1 intuitions in rational discourse. My impression of the most-negative comments was that they could be caricatured not as "auggh, get this fuzzy stuff out of my rational discourse" but as "yikes, cultists and mind-manipulators incoming, run away". Less-caricaturedly: some of that discussion makes me uneasy because it seems as if there is a smallish but influential group of people around here who have adopted a particular set of rather peculiar practices and thought-patterns and want to spread the word about how great they are, but are curiously reluctant to be specific about them to those who haven't experienced them already -- and all that stuff pattern-matches to things I would rather keep a long way away from.
For the avoidance of doubt, the above is not an accusation, pattern-matching is not identity, etc., etc., etc. I mention it mostly because I suspect that uneasiness like mine is a likely source for a lot of the negative reactions, and because it's a very different thing from thinking that the topic in question should somehow be off-limits in rational discourse.
FYI, I've definitely updated that the "fuzzy-system-1 intuitions" not being the concern for most (or at least many) of the critics in Kensho and Circling.
(I do think there's a related thing, though, which is that every time a post that touches upon fuzzy-system-1 stuff spawns a huge thread of intense argumentation, the sort of person who'd like to post that sort of thread ends up experience a chilling effect that isn't quite what the critics intended. In a similar although not quite analogous way that simply having the Reign of Terror option can produce a chilling effect on critics)
I, for one, am not anti-criticism.
I also suspect Ray isn't either, and isn't saying that in his post, but it's a long post, so I might have missed something.
The thing I find annoying to deal with is when discussion is subtly more about politics than the actual thing, which Ray does mention.
I feel like people get upvoted because
I appreciated the criticisms raised in my Circling post, and I upvoted a number of the comments that raised objections.
But the subsequent "arguments" often spiraled into people talking past each other and wielding arguments as weapons, etc. And not looking for cruxes, which I find to be an alarmingly common thing here, to the degree that I suspect people do not in fact WANT their cruxes to be on the table, and I've read multiple comments that support this.
I simply don’t think the concept of the “crux” (as CFAR & co. use it) is nearly as universally applicable to disagreements as you (and others here) seem to imply.
This is surprising to me. A crux is a thing that if you didn't believe it you'd change your mind on some other point -- that seems like a very natural concept!
Is your contention that you usually can't fine any one statement such that if you changed your mind about it, you'd change your mind about the top-level issue? (Interestingly, this is the thrust of top comment by Robin Hanson under Eliezer's Is That Your True Rejection? post.)
Ok, then that's the crux of this argument. Personally, I value Eliezer's writing and Conor Moreton's writing more than I value a culture of unfettered criticism.
This seems like a good argument for the archipelago concept? You can have your culture of unfettered criticism on some blogs, and I can read my desired authors on their blogs. Would there be negative consequences for you if that model were followed?
I think Eliezer has a different set of "reasons a comment might aggravate him" than most of the other authors who've complained to us. (note: I'm not that confident in the following, and I don't want this to turn into a psychoanalyze Eliezer subthread and will lock it if it appears to do that)
I think one of the common failure modes he wants the ability to delete are "comments that tug a discussion sideways into social-reality-space" where people's status/tribal modes kick in, distorting people's epistemics and the topic of the conversation. In particular, comments that subtly do this in such a way that most people won't notice, but the decline becomes inevitable, and there's no way to engage with the discussion that doesn't feed into the problem.
I think looking at his current Facebook Wall (where he deletes things that annoy him) is a pretty reasonable look into what you might expect his comments on LW to look like.
But, speaking of that:
I think an important factor to consider in your calculus is that the end result of the the 2 years of great comments you refer to, was Eliezer getting tired of dealing with bullshit and mov...
I think rationalists systematically overestimate the benefits of public discussion (and underestimate the costs).
When something is communicated one-on-one, the cost is effectively doubled because it consumes the sender's time. I think is roughly offset by a factor of 2 efficiency bonus for tailored/interactive talk relative to static public talk. Even aside from the efficiency of tailored talk, a factor of 2 isn't very big compared to the other costs and benefits.
So I think the advantages of public discussion are not mostly the efficiency from many people being able to read something that got written once. Instead they are things like:
Given that rationalists estimate the value of public discourse (and also private discourse) higher than almost anyone else, it is certainly plausible that we overestimate it. Given I estimate it higher than most rationalists, it's even more plausible that I overestimate it. But if anything, I think we underestimate it, and Paul points to some of the reasons why.
Paul is implicitly, I believe, employing a sort of 'student-teacher' model of communication; the teacher transmits communication, the student learns it. The customization and lack of politicization in person is about a factor of two better, which then roughly 'cancels out' the increased cost. But if that's true, it's then pointed out that there are a number of other key advantages to public discourse. Paul's extra reasons seem super important, especially if we add creation of common knowledge and vocabulary to the last one. He also points out that teacher time is typically more valuable than student time, meaning the cost ratio is more than two, perhaps much more. And time spent writing dwarfs the time each person spends reading, so that further increases the ratio.
So it seems like e...
Zvi already mentioned this, but I just want to emphasize that I think error finding and correction (including noticing possible ambiguities and misunderstandings) is one of the most important functions of discussion, and it works much better in a group discussion (especially for hard to detect errors) because everyone interested in the discussion can see the error as soon as one person notices it and points it out. If you do a series of one-on-one discussions, to achieve the same effect you'd have to track down all the previous people you talked to and inform them. I think in practice very few people would actually bother to do that, due to subconscious status concerns among other reasons, which would lead to diverging epistemic states among people thinking about the topic and ensuing general confusion.(Sometimes an error is found that the "teacher" doesn't recognize as an error due to various cognitive biases, which would make it impossible to track down all the previous discussants and inform them of the error.)
Zvi's "asynchronous and on demand" is also a hugely important consideration for me. In an one-on-one discussion, the fact that I can ask questions and ...
Broadcasting is good for transferring information from people with more valuable time to people with less valuable time---in those cases "using up the explainer's time" much more than doubles the cost. I expect it would be better to do transmitting information 1-on-1 transmission to specialized explainers who broadcast it, but we aren't good at that.
This suggests some models:
(where '=>' implies telling a lot of people via an essay)
These models are in order of decreasing net cost to the system, if we assume the thinker's time is the most expensive.
Let me propose another model
I think there's room for LW to intervene on this part of the model. Here are some ways to do that:
Richard_Kennaway's summary of the PNSE paper on the Kensho thread is the most valuable thing I've ever read about meditation. However, it's critical of meditation. If Valentine had specified a comment policy saying "please don't be critical of meditation, we are looking for 201 level discussions and beyond", Richard might have written a top-level post instead. But Richard has zero top-level posts to date, so it's more likely that he just wouldn't bother.
the most valuable thing I've ever read about meditation. However, it's critical of meditation.
Possibly obvious, but seems worth noting explicitly: that paper discusses the result of some very advanced meditative states, which usually need to be pursued with serious intent. It shouldn't be considered a criticism of meditation as a whole, given that there are a lot of benefits that typically show up much before one gets to the realm of non-symbolic stuff and which are more objectively verifiable. Also, not all meditative practices even have non-symbolic states as their goal.
Note also that even in that paper, the disconnect between internal experience and actual externally-reported signs of bodily/emotional awareness was only reported in three interviewees out of 50, and that the memory loss stuff only started showing up around the last stage, which several traditions were noted to stop short of.
(I also suspect that there may have been a bit of a disconnect in the language used by the interviewer and the interviewees: I think I might have had a few brief glimpses of what a non-symbolic state feels like, and one thing in particular that I would still have emotions as ...
Yeah. The chilling effect is almost entirely (I think) about people worrying they will be censored, rather than actual censorship. I have full ability on my blog to delete comments, have used it only for removing typo/duplicate comments, and felt vaguely bad about even doing that. Val censoring that comment would be a huge surprise and forced update for me, but Richard thinking he might censor it seems a reasonable thing to worry about.
Of course, if it was censored, there's nothing to stop someone from then posting it elsewhere or as its own post, if they think it's valuable, and presumably the system will save the comment so you can copy it. Given that, this circles back to the general case of The Fear people have around posting and commenting, as if negative feedback of any sort was just awful. If you get censored, it's not that big a deal. Might even be good, now you know where the line is and not to do that again!
I'm confused about what sort of content belongs on LW 2.0, even in the Archipelago model.
I've been a lurker on LW and many of the diaspora rational blogs for years, and I've only recently started commenting after being nudged to do so by certain life events, certain blog posts, and the hopeful breath of life slightly reanimating LessWrong.
Sometimes I write on a personal blog elsewhere, but my standards are below what I'd want to see on LW. Then again, I've seen things on LW that are below my standards of what I expect on LW.
I've seen it said multiple times that people can now put whatever they want on their personal LW spaces/blogposts, and that's stressed again here. But I still feel unsettled and like I don't really understand what this means. Does it mean that anyone on the internet talking about random stuff is welcome to have a blog on LW? Does it mean well known members are encouraged to stick around and can be off the rationality topic in their personal blogposts? How about the unknown members? How tangential can the topic be from rationality before it's not welcome?
Could a personal post about MealSquares and trading money for ti...
The idea is indeed that you are welcome to post about whatever you want on LW, and as we get more and more content, we will make people's personal blogs less visible from the frontpage, and instead add subscription mechanisms that allow people to subscribe to the specific people they want to follow (which they will see in addition to the frontpage discussion).
We are planning to turn off the ability to lose and gain global-karma for personal blogposts in the near future, though we are still planning to allow people to upvote and downvote content (though we might put a lower bound of something like -4 or 0 on the negative score a post can get). So as soon as that happens, you will no longer be able to lose karma from writing a post on your personal blog that people didn't like.
We are still optimizing the site for the people who are trying to make progress on rationality and various related topics, and so while it's possible to use LessWrong as a fashion blog, you will probably find the feature set of the site not super useful to do that, and you won't benefit super much from doing that on LessWrong over something like Medium (unless you want to analyze fashion u...
An issue I currently notice with Personal Blogposts is that they serve two purposes, which are getting conflated:
1) blogposts that don't meet the frontpage guidelines (i.e. touching upon politics, or certain kinds of ingroupy stuff), but which you expect to be worth the time and attention of people who are heavily involved with the community.
2) blogposts that you aren't making a claim are worth everyone's attention.
Right now there's a fair amount of posts of type #1, which means if you want to stay up to date on them, you need to viewing all posts. But that means seeing a lot of posts in type #2, that the author may well have preferred not to force into your attention unless you already know the author and subscribing to them. But they don't have a choice.
I predict we'll ultimately solve that by splitting those two use cases up.
Being worthy of everyone's attention is quite the bar! I certainly wouldn't want to only publish things that rise to the level of 'everyone or at least a large percentage of rationalists should read this post.' The majority of my posts do not rise to that level, and by math almost no posts in the world can rise to that level.
The global justification is, if you don't let me put my third-tier posts on my personal blog here, then I'll be creating content that doesn't end up on LW2.0, which means that to read all my stuff they have to check my blog itself at DWATV, which means they get into the habit of reading me there and LW2.0 fails as a central hub. You want to make it possible for people to just not check other sources, at all.
Once again, I just want to say a huge thanks to the team building the site. This type of work requires really careful consideration of tradeoffs, is hugely leveraged, and almost automatically displeases some people by the tradeoffs made.
For my part, I don't have any real opinions on the moderation guidelines and tech buildout here specifically; I just wanted to say thanks and salut for all the thoughtful work going into it. It's too often thankless work, but it makes a tremendous difference. Regards and appreciation.
Instead of purely focusing on whether people will use these powers well, perhaps we should also talk about ways to nudge them towards responsibility?
One thing this has made me realize is that I think people are explicitly assuming Reign of Terror means "access to Delete-And-Hide". Which is not actually the case right now, but which is a reasonable assumption and we should probably be designing the site around that being intuitively obvious.
(right now, Reign of Terror is just a text description, informing commenters what to expect. But it probably makes sense for users to only have Delete-And-Hide if their moderation style is set to Reign of Terror, and/or if they use the delete-and-hide feature, the give post's moderation setting is changed to Reign of Terror regardless of what their usual setting is)
Delete-and-Hide requiring higher karma that other delete options also seems reasonable to me. (And regardless of whether it should or not, we should probably refactor the code such that it's handled separately from from the block-of-permissions that grants the other deletes, so we have more control over it)
I'm a huge fan of the Archipelago model, but I'm unsure how well our current structure is suited to it. On Reddit, you only have to learn one set of moderation norms per sub. On Less Wrong, learning one set of moderation norms per author seems like a much higher burden.
In fact, the Archipelago model itself, is one set of norms per group. If you don't like the norms of a particular group, you just don't go there. This is harder when there isn't a clear separation and everyone's posts are all mixed together.
This behavior of writing a post and getting unhelpful comments: is it something that can be changed by tweaking the karma system?
Like, right now, if I read a post and think of a true-but-unhelpful objection, maybe I post the objection in the hope of getting upvotes.
But maybe if you make the post author's upvotes worth more than upvotes by random schmoes, then I optimize more for posting things the post author will like?
Something that has only just now occurred to me, which in retrospect is perfectly consistent with my other experiences, is the extent to which posts like this help to improve my sense of how to contribute. Specifically, articulating the intuitions behind the decisions is very valuable to me, quite aside from the actual administrative direction of the website. This is the most valuable meta-discussion chain I have observed.
I support the current direction; I add that this will help keep the site from growing stale; I have no concerns about unjust infringement of expression. Good show!
Some quick thoughts:
1) I realize that this may be some work, but maybe "down" votes could be more specific. Some categories I've seen have been:
I would assume that if there are a few things you are trying to discourage, having the ability for people to label those things would be a good step to measuring and minimizing...
Thanks for articulating why Facebook is a safer and more pleasant place to comment than LW. I tried to post pretty much this on a previous thread but wasn't able to actually articulate the phenomenon so didn't say anything.
That being said, I still feel like I'd rather just post on Facebook.
There are two specific problems with Facebook as a community forum that I'm aware of. The first is that the built-in archiving and discovery tools are abysmal, because that's not the primary use case for the platform. Fortunately, we know there...
There are two specific problems with Facebook as a community forum that I'm aware of. The first is that the built-in archiving and discovery tools are abysmal, because that's not the primary use case for the platform. Fortunately, we know there's a technical solution to this, because Jeff Kaufman implemented it on his blog.
I don't understand this response. That there exists a solution doesn't mean that there exists a solution that 1) is easy to use and 2) that people will actually use. One of the many advantages of hosting a conversation on a blog post instead of on a Facebook status is that it's easy for random people to link to that blog post years later. Even if people could in principle do this for Facebook statuses with the appropriate tools, the trivial inconveniences are way too high and they won't.
(I've already had one friend explicitly say that he was looking for a Facebook status I wrote because he wanted to show it to someone else but found it too annoying to look for and gave up.)
Other than that, Facebook seems to have the whole "archipelago" thing pretty much solved.
I actually think there's a deep sense in which Facebook has not accomplished archipelago, and an additional deep sense in which they have not accomplished public archipelago.
Archipelago doesn't just mean "you've filter-bubbled yourself such that people you only hang out with likeminded people." It means you've filtered yourself and then used that filtering to enforce norms that you wouldn't be able to enforce otherwise, allowing you to experiment with culture building.
On FB, I've seen a small number of people do this on purpose. Mostly I see people sort of halfheartedly complaining about norms, but neither setting explicit norms for people to follow nor following through on kicking people out if they don't. (An issue is that FB is designed to be a melting pot. Your mom, your college friends, and rationalist friends are all bumping into each other, and have different assumptions about what norms even mean)
And then, re Public Archipelago: Facebook very much works against the ability for good ideas to bubble up into a central conversation that everyone can be aware of. You could attempt to solve this by building around Facebook, but Facebook really doesn't want you to do that and it's a pain.
I expect to have serious comments after some reflection, but I wanted to register that this is extraordinarily well-thought out.
Also, what a perfect post for the day I reach 2k!
Personal Blogposts where all commenters are only allowed to speak in poetry.
Challenge accepted.
How do I put it, so as not to offend anyone... I think this is the right discussion for me to say that although I percieve this comment as positive, this definitely is not one I would wish to allocate my attention to, given the choice. I would have expected such posts to get downvoted. I suggest two separate systems of voting: one for positive fuzzy feelings, one for worthiness of attention. What I hope is that it would mitigate the reluctance to downvote (or to not upvote) stemming from the social nature of humans. I.e. we could continue not discouraging each other while still having a useful conversation.
I strongly agree about the importance of play in idea generation. There's a scale with yes, and at one end and well, actually at the other, and I try to stay close to the "yes, and" end all else equal. (Though if I see someone defecting on good conversation norms, I'm sometimes willing to defect in response.)
Re: trust, academia has a nice model here: collaborate with friends at your university, and eventually publish & make your ideas known to strangers at other universities. From an epistemic hygiene perspective, it feels right ...
On Frontpage posts, we will want to have some kind of standard, which I’m not sure we can formally specify. We’re restricting moderation tools to users with high karma, so that only people who’ve already internalized what LessWrong is about have access to them. We want experimentation that productively explores rational-discussion-space.
This policy doesn't currently make sense to me, but it might just be because I don't understand some of the mechanics of how the site works.
Am I correct that currently frontpage posts can be created in two ways: e...
The idea that one does not simply comment without knowing the moderation policy seems like an error. That doesn't mean that there's no value in knowing the moderation policy, but if 20 posts had 20 different moderation policies and you wanted to write comments, the likely effect is still you write 20 comments and nothing happens to any of them; if 1-2 of them do get minimized, maybe then you look at what happened. Or alternatively, you'd check if and only if you know you're in a grey area.
I do notice that I might look at the comment policy when deciding whether to read the comments...
it is quite a bit of cognitive overhead for the average commenter to have to memorize the moderation norms of 20 different personal blogs
yeah i'm worried about this...
would like it if there were some obvious indication when someone's moderation policy significantly deviates from what one would typically expect, such that I will definitely be sure to read theirs if it's unusual (like the poetry one).
and otherwise, I'd want to encourage ppl to stick to the defaults...
Also seems like, given LW's structure, it makes way more sense to have "moderation policies for posts" and not "moderation policies for blogs / authors." I don't really see the blogs. I see the posts. I really can't distinguish very well between blogs. So I'm going to check post-level moderation policy and not really track blog-level / author-level moderation policy.
And as an author, I may want each of my posts to have different policies anyway, so I might change them for each post... I dunno how that works right now.
This is a bit of a tangent, but have LessWrongers considered replacing the norm of using common names in sequential order like 'Alice, Bob, Carl...' etc? They could be replaced by less common names. Sometimes I start thinking of rationality community members when their names are also used for stand-in people in examples in blog posts.
I think the "not written down at all" link is broken. It's taking me to "https://www.lesserwrong.com/posts/.../writing-down-conversations", which isn't a valid link.
While the comment threads about both Kensho and Circling had a lot of conflict, I don't think that's bad. It's conflict that worth to be had and it's good to be open about the lines of that conflict in a public forum.
While the posts haven't changed my opinions about the main stance about the topics they did give me a better idea about the inferential gap.
At the moment I have three drafts:
1) The epistemology of NLP
2) What does it mean to be in relationship?
3) Pierget's Assimilation/Accommodation/Equilibration
It's conflict that worth to be had and it's good to be open about the lines of that conflict in a public forum.
That's not my crux. The conflict on the Kensho and Circling threads made me substantially more worried about posting similar threads. I don't enjoy the thought of being punished for trying to introduce new ideas to LW by having to spend my time dealing with demon threads. This is the sort of thing that made me stop writing on LW 1.0, and it's also an invisible cost; you won't notice it's happening unless you talk to people who aren't posting about why they aren't posting.
This has change my thinking quite a bit around how to create an anti-fragile environment that still has strong norms and values. I would love to see more work in this direction and think about the "Archipelago Model" frequently
I thought about the idea of removing the effect of karma votes on personal blogs in the future and I don't think it's a good idea.
I do understand the lure of wanting to be inclusive of any kind of writing and also allow those people who's posts currently get downvoted into oblivion but if a lot of personal blogs are of that quality less people would read personal blogs of people they didn't subscribe to. This inturn will mean that if a new person writes a personal blog post, nobody will read it.
Longer-term I think it's better to make the software easy to install by other people so that the people who do want to have a LessWrong2.0 style blog about cat pictures can install their own instances.
My perspective, Ray might have different thoughts on this:
5 years doesn't strike me as insane. It seems that most online platforms require a makeover about once every 5 years, so yeah, if this goes well then launching LessWrong 3.0 in 2022-2023 seems quite reasonable. Though some platforms seem to last 10 years without makeover before they seriously decline, so maybe we can put it off until 2027-2028. I would be surprised if this website would be successful for more than 10 years without significant rework (the internet changes quickly, and there is a good chance we will all be browsing in VR by then, or social networks will have wholly consumed the whole internet, or some other change of the scale of the onset of social networks happens).
Yeah, it's both important to me that the people I see doing the most valuable work on rationality and existential risk feel comfortable posting to the platform, and that we can continue replacing the people we will inevitably lose because of natural turnover with people of equal or better quality.
This has definitely not been the case over the previous 3 years of LessWrong, and so to fix that we will require some changes. My diagnosis of why that happened is partially that the nature of how people use the internet changed (with the onset of social networks and more competition due to better overall technology), partially because the people who were doing good work on the problems changed, and partially because we simply didn't have a system that could productively fight against the forces of entropy for too long, and so a lot of the best people left.
I agree that it is hard to deal with large numbers of people joining at the same time, which is why I am indeed not super interested in discontinuous growth and am not pushing for anything in that direction. I do still think we are under the number of people who can productively talk to each other, and that at this point in time further-sustained, slow growth is valuable and net-positive.
Is your concern that it's not clear whether PDV's estimate is a mean, median, or mode? "Median and mode" seems like a reasonable guess (though have to be careful when defining mode etc.)
Being ambiguous about your prediction leaves wiggle room, but that's typical for english sentences; giving an estimate without saying what exactly it means is still less ambiguous than the default.
The recent moderation tools announcement represents a fairly major shift in how the site admins are approaching LessWrong. Several people noted important concerns about transparency and trust.
Those concerns deserve an explicit, thorough answer.
Summary of Concepts
i. The Problem
The issue with LessWrong that worries me the most:
In the past 5 years or so, there’s been a lot of progress – on theoretical rationality, on practical epistemic and instrumental rationality, on AI alignment, on effective altruism. But much of this progress has been on some combination of:
People have complained about this. I think a common assumption is something like “if we just got all the good people back on LessWrong at the same time you’d have a critical mass that could reboot the system.” That might help, but doesn't seem sufficient to me.
I think LW2.0 has roughly succeeded at becoming “the happening place” again. But I still know several people who I intellectually respect, who find LessWrong an actively inhospitable place and don’t post here, or do so only grudgingly.
More Than One Way For Discussion To Die
I realize that there’s a very salient pathway for moderators to abuse their power. It’s easy to imagine how echo chambers could form and how reign-of-terror style moderation could lead to, well, reigns of terror.
It may be less salient to imagine a site subtly driving intelligent people away due to being boring, pedantic, or frustrating, but I think the latter is in fact more common, and a bigger threat to intellectual progress.
The current LessWrong selects somewhat for people who are thick skinned and conflict prone. Being thick-skinned is good, all being equal. Being conflict prone is not. And neither of these are the same as being able to generate useful ideas and think clearly, the most important qualities to cultivate in LessWrong participants.
The site admins don’t just have to think about the people currently here. We have to think about people who have things to contribute, but don’t find the site rewarding.
Facebook vs LessWrong
When I personally have a new idea to flesh out... well...
...I’d prefer a LessWrong post over a Facebook post. LW posts are more easily linkable, they have reasonable formatting options over FB’s plain text, and it’s easier to be sure a lot of people have seen it.
But to discuss those ideas…
In my heart of hearts, if I weren’t actively working on the LessWrong team, with a clear vision of where this project is going... I would prefer a Facebook comment thread to a LessWrong discussion.
There are certain blogs – Sarah, Zvi, Ben stick out in my mind, that are comparably good. But not many – the most common pattern is “post idea on blog, and the good discussion happens on FB, and individual comment insights only make it into the broader zeitgeist if someone mentions them in a high profile blogpost."
On the right sort of Facebook comment thread, at least in my personal filter bubble, I can expect:
Beyond that, more subtle: even if I don’t know everyone, an intellectual discussion on FB usually feels like, well, we’re friends. Or at least allies.
Relatedly: the number of commenters is manageable. The comments on Slatestarcodex are reasonably good these days, but… I’m just not going to sift through hundreds or thousands of comments to find the gems. It feels like a firehose, not a conversation.
Meanwhile, the comments on LessWrong often feel... nitpicky and pointless.
If an idea isn’t presented maximally defensibly, people will focus on tearing holes in the non-loading-bearing parts of the idea, rather than help refine the idea into something more robust. And there’ll be people who disagree with or don’t understand foundational elements that the idea is supposed to be building off of, and the discussion ends up being about rehashing 101-level things instead of building 201-level knowledge.
Filter Bubbles
An obvious response to the above might be “of course you prefer Facebook over LessWrong. Facebook heavily filter bubbles you so that you don’t have to face disagreement. It’s good to force your ideas to intense scrutiny.”
And there’s important truth to that. But my two points are that:
My experience is that my filter bubbles (whether on FB, Google Docs or in-person) do involve a lot of disagreement, and the disagreement is higher quality. When someone tells me I’m wrong, it’s often accompanied by an attempt to understand what my goals are, or what the core of a new idea was, which either lets me fix an idea, or abandon it but find something better to accomplish my original intent.
(On FB, this isn’t because the average commenter is that great, but because of a smallish number of people I deeply respect, who have different paradigms of thinking, at least 1-2 of whom will reliably show up)
There seems to be a sense that good ideas form fully polished, without any work to refine them. Or that until an idea is ready for peer review, you should keep it to yourself? Or be willing to have people poke at it with no regard how hedonically rewarding that experience is? I’m not sure what the assumption is but it’s contrary to how everyone I personally know generates insights.
The early stages work best when playful and collaborative.
Peer review is important, but so is idea formation. Idea formation often involves running with assumptions, crashing them into things and seeing if it makes sense.
You could keep idea-formation private and then share things when they’re ‘publicly presentable’, but I think this leads to people tending to keep conversation in “safe, private” zones longer than necessary. And meanwhile, it’s valuable to be able to see the generation process among respected thinkers.
Public Discussion vs Knowledge Building
Some people have a vision of Less Wrong as a public discussion. You put your idea out there. A conversation happens. Anyone is free to respond to that conversation as long as they aren’t being actively abusive. The best ideas rise to the top.
And this is a fine model, that should (and does) exist in some places. But:
These might seem like the same goal. And I share an aesthetic sense that in the ‘should’ world, where things are fair, public discussion and knowledge-building are somehow the same goal.
But we don’t live in the ‘should’ world.
We live in the world where you get what you incentivize.
Yes, there’s a chilling effect when authors are free to delete comments that annoy them. But there is a different chilling effect when authors aren’t free to have the sort of conversation they’re actually interested in having. The conversation won’t happen at all, or it’ll happen somewhere else (where you can't comment on their stuff anyway).
A space cannot be universally inclusive. So the question is: is LessWrong one space, tailored for only the types of people who enjoy that space? Or do we give people tools to make their own spaces?
If the former, who is that space for, and what rules do we set? What level of knowledge do we assume people must have? We’ve long since agreed “if you show up arguing for creationism, this just isn’t the space for you.” We’ve generally agreed that if you are missing concepts in the sequences, it’s your job to educate yourself before trying to debate (although veterans should politely point you in the right direction).
What about posts written since the sequences ended?
What skills and/or responsibilities do we assume people must have? Do we assume people have the ability to notice and speak up about their needs a la Sarah Constantin’s Hierarchy of Requests? Do we require them to be able to express those needs ‘politely’? Whose definition of polite do we use?
No matter which answer you choose for any of these questions, some people are going to find the resulting space inhospitable, and take their conversation elsewhere.
I’d much rather sidestep the question entirely.
A Public Archipelago Solution
Last year I explored applying Scott Alexander's Archipelago idea towards managing community norms. Another quick recap:
And then...
Taking this a step farther is the idea of Public Archipelago, with islands that overlap.
Let people create their own spaces. Let the conversations be restricted as need be, but centralized and public, so that everyone at least has the opportunity to follow along, learn, respond and build off of each other’s ideas, instead of having to network their way into various social/internet circles to keep up with everything.
This necessarily means that not all of LessWrong will be a comfortable place to any given person, but it at least means a wider variety of people will be able to use it, which means a wider variety of ideas can be seen, critiqued, and built off of.
Healthy Disagreement
Now, there’s an obvious response to my earlier point about “it’s frustrating to have to explain 101-level things to people all the time.”
Maybe you’re not explaining 101-level things. Maybe you’re actually just wrong about the foundations of your ideas, and your little walled garden isn’t a 201 space, it’s an echo chamber built on sand.
This is, indeed, quite a problem.
It’s an even harder problem than you might think at first glance. It’s difficult to offer an informed critique of something that’s actually useful. I’m reminded of Holden Karnofsky’s Thoughts on Public Discourse:
The obvious criticisms of an idea may have obvious solutions. If you interrupt a 301 discussion to ask “but have you considered that you might be wrong about everything?”... well, yes. They have probably noticed the skulls. This often feels like 2nd-year undergrads asking post-docs to flesh out everything they’re saying, using concepts only available to the undergrads.
Still, peer review is a crucial part of the knowledge-building process. You need high quality critique (and counter-critique, and counter-counter-critique). How do you square that with giving an author control over their conversation?
I hope (and fairly confidently believe) that most authors, even ones employing Reign-of-Terror style moderation policies, will not delete comments willy nilly – and the site admins will be proactively having conversations with authors who seem to be abusing the system. But we do need safeguards in case this turns out to be worse than we expect.
The answer is pretty straightforward: it’s not at all obvious that the public discussion of a post has to be on that particular post’s comment section.
(Among other things, this is not how most science works, AFAICT, although traditional science leaves substantial room for improvement anyhow).
If you disagree with a post, and the author deletes or blocks you from commenting, you are welcome to write another post about your intellectual disagreement.
Yes, this means that people reading the original post may come away with an impression that a controversial idea is more accepted than it really is. But if that person looks at the front page of the site, and the idea is controversial, there will be both other posts and recent comments arguing about its merits.
It also means that no, you don’t automatically get the engagement of everyone who read the original post. I see this as a feature, not a bug.
If you want your criticism to be read, it has to be good and well written. It doesn’t have to fit within the overall zeitgeist of what’s currently popular or what the locally high-status people think. Holden’s critical Thoughts on Singularity Institute is one of the most highly upvoted posts of all time. (If anything, I think LessWrong folk are too eager to show off their willingness to dissent and upvote people just for being contrarian).
It does suck that you must be good at writing and know your audience (which isn’t necessarily the same as good at thinking). But this applies just as much to being the original author of an idea, as to being a critic.
The author of a post doesn’t owe you their rhetorical strength and audience and platform to give you space to write your counterclaim. We don’t want to incentivize people to protest quickly and loudly to gain mindshare in a popular author’s comment section. We want people to write good critiques.
Meanwhile, if you're making an effort to understand an author's goals and frame disagreement in a way that doesn't feel like an attack, I don't anticipate this coming up much in the first place.
ii. Expectations and Trust
I think a deep disagreement that underlies a lot of the debate over moderation: what sort of trust is important to you?
This is a bit of a digression – almost an essay unto itself – but I think it’s important.
Elements of Trust
Defining trust is tricky, but here’s a stab at it: “Trust is having expectations of other people, and not having to worry about whether those expectations will be met.”
This has a few components:
Which expectations?
You might trust people…
Trust is a multiple-place function. Maybe you trust Alice to reliably provide all the relevant information even if it makes her look bad. You trust Bob to pay attention to your emotional state and not say triggering things. You can count on Carl to call you on your own bullshit (and listen thoughtfully when you call him on his). Eve will reliably enforce her rules even when it’s socially inconvenient to do so.
You may care about different kinds of trust in different contexts.
How much do you trust a person or space?
For the expectations that matter most to you, do you generally expect them to be fulfilled, or do you have to constantly monitor and take action to ensure them?
With a given person, or a particular place, is your guard always up?
In high trust environments, you expect other people to care about the same expectations you do, and follow through on them. This might mean looking out for each other’s interests. Or, merely that you’re focused on the same goals such that “each other’s interests” doesn’t come into play.
High trust environments require you to either personally know everyone, or to have strong reason to believe in the selection effects on who is present.
Examples:
Low trust environments are where you have no illusions that people are looking out for the things you care about.
The barriers to entry are low. People come and go often. People often represent themselves as if they are aligned with you, but this is poor evidence for whether they are in fact aligned with you. You must constantly have your guard up.
Examples:
Transparent Low Trust, Curated High Trust
Having to watch your back all the time is exhausting, and there’s at least two strategy-clusters I can think of to alleviate that.
In a transparent low trust environment, you don’t need to rely on anyone’s word or good intentions. Instead, you rely upon transparency and safeguards built into the system.
It’s your responsibility to make use of those safeguards to check that things are okay.
A curated high trust environment has some kind of strong barrier to entry. The advantage is that things can move faster, be more productive, require less effort and conflict, and focus only on things you care about.
It’s the owner of the space’s responsibility to kick people out if they aren’t able to live up to the norms in the space. It’s your responsibility to decide whether you trust the the space, and leave if you don’t.
The current atmosphere at LessWrong is something like “transparent medium trust.” There are rough, site-level filters on what kind of participation is acceptable – much moreso than the average internet hangout. But not much micromanaging on what precise expectations to uphold.
I think some people are expecting the new moderation tools to mean “we took a functioning medium trust environment and made it more dangerous, or just weirdly tweaked it, for the sake of removing a few extra annoying comments or cater to some inexplicable whims.”
But part of the goal here is to create a fundamental phase shift, where types of conversations are possible that just weren't in a medium-trust world.
Why High Trust?
Why take the risk of high trust? Aren’t you just exposing yourself to people who might take advantage of you?
I know some people who’ve been repeatedly hurt, by trying to trust, and then having people either trample all over their needs, or actively betray them. Humans are political monkeys that make up convenient stories to make themselves look good all the time. If you aren’t actually aligned with your colleagues, you will probably eventually get burned.
And high trust environments can’t scale – too many people show up with too many different goals, and many of them are good at presenting themselves as aligned with you (they may even think they’re aligned with you), but… they are not.
LessWrong (most likely) needs to scale, so it’s important for there to be spaces here that are Functioning Low Trust, that don’t rely on load-bearing authority figures.
I do not recommend this blindly to everyone.
But. To misquote Umesh – "If you’re not occasionally getting backstabbed, you’re probably not trusting enough."
If you can trust the people around you, all the attention you put into watching your back can go to other things. You can expect other people to look out for your needs, or help you in reliable ways. Your entire body physiologically changes, no longer poised for fight or flight. It’s physically healthier. In some cases it’s better for your epistemics – you’re less defensive when you don’t feel under attack, making it easier to consider opposing points of view.
I live most of my life in high trust environments these days, and… let me tell you holy shit when it works it is amazing. I know a couple dozen people who I trust to be honest about their personal needs, to be reasonably attentive to mine, who are aligned with me on how to resolve interpersonal stuff as well as Big Picture How the Universe Should Look Someday.
When we disagree (as we often do), we have a shared understanding of how to resolve that disagreement.
Conversations with those people are smooth, productive, and insightful. When they are not smooth, the process for figuring out how to resolve them is smooth or at least mutually agreed upon.
So when I come to LessWrong, where the comments assume at-most-medium trust… where I’m not able to set a higher or different standard for a discussion beyond the lowest common denominator...
It’s really frustrating and sad, to have to choose between a public-untrusted and private-but-high-trust conversation.
It’s worth noting: I participate in multiple spaces that I trust differently. Maybe I wouldn’t recommend particular friends join Alice’s space because, while she’s good stating her clear reasons for things and evaluating evidence clearly and making sure others do the same, she’s not good at noticing when you’re triggered and pausing to check in if you’re okay.
And maybe Eve really needs that. That’s usually okay, because Eve can go to Bob’s space, or run her own.
Sometimes, Bob’s space doesn’t exist, and Eve lacks the skills to attract people to a new space. This is really important and sad. I personally expect LessWrong to contain a wide distribution of preferences that can support many needs, but it probably won’t contain something for everyone.
Still, I think it’s an overall better strategy to make it easier to create new subspaces than to try to accommodate everyone at once.
Getting Burned
I expect to get hurt sometimes.
I expect some friends (or myself) to not always be at our best. Not always self-aware enough to avoid falling into sociopolitical traps that pit us against each other.
I expect that at least some of the people I’m currently aligned with, I may eventually turn out to be unaligned with, and to come into conflict that can’t be easily resolved. I’ve had friendships that turned weirdly and badly adversarial and I spent months stressfully dealing with it.
But the benefits of high trust are so great that I don’t regret for a second having spent the first few years with those friends in a high-trust relationship.
I acknowledge that I am pretty privileged in having a set of needs and interpersonal preferences that are easier to fit into a high trust environment. There are people who just don’t interface well with the sort of spaces I thrive in, who may never get the benefits of high trust, and that... really sucks.
But the benefit of the Public Archipelago model is that there can be multiple subsections of the site with different norms. You can participate in discussions where you trust the space owner. Some authors may clearly spell out norms and take the time to clearly explain why they moderate comments, and maybe you trust them the most.
Some authors may not be willing to take that time. Maybe you trust them less, or maybe you know them well enough that you trust them anyhow.
In either case, you know what to expect, and if you’re not okay with it, you either don’t participate, or respond elsewhere, or put effort into understanding the author’s goals so that you are able to write critiques that they find helpful.
iii. The Fine Details
Okay, but can’t we at least require reasons?
I don’t think many people were resistant to deleting comments – the controversial feature was “delete without trace.”
First, spam bots, and dedicated adversaries with armies of sockpuppets make it at least necessary for this tool to be an available (LW2.0 has had posts with hundreds of spam or troll comments we quietly delete and IP ban)
For non-obvious spam…
I do hope delete without trace is used rarely (or that authors send the commenter a private reason when doing so). We plan to implement the moderation log Said Achmiz recommended, so that if someone is deleting a lot of comments without trace you can at least go and check, and notice patterns. (We may change the name to “delete and hide”, since some kind of trace will be available).
All things being equal, clear reasons are better than none, and more transparency is better than less.
But all things are not equal.
Moderation is work.
And I don’t think everyone understands that the amount of work varies a lot, both by volume, and by personality type.
Some people get energized and excited by reading through confrontational comments and responding.
Some people find it incredibly draining.
Some people get maybe a dozen comments on their articles a day. Some get barely any at all. But some authors get hundreds, and even if you’re the sort of person who is energized by it, there are only so many hours in a day and there are other things worth doing.
Some comments are not just mean or dumb, but immensely hateful and triggering to the author, and simply glancing at a reminder that it existed is painful – enough to undo the personal benefit they got from having written their article in the first place.
For many people, figuring out how to word a moderation notice is stressful, and I’m not sure whether it’s more intense on average to have to say:
vs
Not to mention that moderation often involves people getting angry at you, so you don't just have to come up with the initial posted reason, but also deal with a bunch of followup that can wreck your week. Comments that leave a trace invite people to argue.
Moderation can be tedious. Moderation can be stressful. Moderation is generally unpaid. Moderators can burn out or decide “you know what, this just isn’t worth the time and bullshit.”
And this is often the worst deal for the best authors, since the best authors attract more comments, and sometimes end up acquiring a sort of celebrity status where commenters don’t quite feel like they’re people anymore, and feel justified (or even obligated) to go out of their way to take them down a peg.
If none of this makes sense to you, if you can’t imagine moderating being this big a deal… well... all I can say is it just really is a god damn big deal. It really really is.
There is a tradeoff we have to make, one way or another, on whether we want to force our best authors to follow clear, legible procedures, or to write and engage more.
Requiring the former can (and has) ended up punishing the latter.
We prioritized building the delete-and-hide function because Eliezer asked for it and we wanted to get him posting again quickly. But he is not the only author to have asked and expressed appreciation for it.
Incentivizing Good Ideas and Good Criticism
I’ll make an even stronger claim here: punishing idea generation is worse than punishing criticism.
You certainly need both, but criticism is easier. There might be environments where there isn’t enough quantity or quality of critics, but I don’t think LessWrong is one of them. Insofar as we don’t have good enough criticism, it’s because the critiques are nitpicky and unhelpful instead of trying to deeply understand unfamiliar ideas and collaboratively improve their load-bearing cruxes.
And meanwhile, I think the best critics also tend to be the best idea-generators – the two skills are in fact tightly coupled – so making LessWrong a place they feel excited to participate in seems very important.
It’s possible to go too far in this direction. There are reasonable cases for making a different tradeoffs that different corners of the internet might employ. But our decision on LessWrong is that authors are not obligated to put in that work if it’s stressful.
Overton Windows, and Personal Criticism
There’s a few styles of comments that reliably make me go “ugh, this is going to become a mess and I really don’t want to deal with it.” Comments whose substance is “this idea is bad, and should not be something LessWrong talks about.”
In that moment, the conversation stops being about whatever the idea was, and starts being about politics.
A recent example is what I’d call “fuzzy system 1 stuff.” The Kensho and Circling threads felt like they were mostly arguing about “is it even okay to talk about fuzzy system 1 intuitions in rational discourse?”. If you wanted to talk about the core ideas and how to use them effectively, you had to wade through a giant, sprawling demon thread.
Now, it’s actually pretty important whether fuzzy system 1 intuitions have a place in rational discourse. It’s a conversation that needs to happen, a question that probably has a right answer that we can converge on (albeit a nuanced one that depends on circumstances).
But right now, it seems like the only discussion that’s possible to have about them is “are these in the overton window or not?”. There needs to be space to explore ideas that aren’t currently in the accepted paradigm.
I’d even claim that doing that productively is one of the things rationality is for.
Similar issues abound with critiquing someone’s tone, or otherwise critiquing a person rather than an idea. Comments like that tend to quickly dominate the discussion and make it hard to talk about anything else. In many cases, if the comment were a private message, it could have been taken as constructive criticism instead of a personal attack that enflares people’s tribal instincts.
For personal criticism, I think the solution is to build tools that make private discussion easier.
For Overton Window political brawls, I think the brawl itself is inevitable (if someone wants to talk about a controversial thing, and other people don’t want them to talk about the controversial thing, you can’t avoid the conflict). But I think it’s reasonable for authors to say “if we’re going to have the overton discussion, can we have it somewhere else? Right here, I’m trying to talk about the ramifications of X if Y is true.”
Meanwhile, if you think X or Y are actively dangerous, you can still downvote their post. Instead of everyone investing endless energy in multiple demon threads, the issue can be resolved via a single thread, and the karma system.
I don’t think this would have helped with the most recent thread, but it’s an option I’d want available if I ever explored a controversial topic in the future.
iv. Towards Public Archipelago
This is a complicated topic, the decision is going to affect people. If you’re the sort of person for whom the status quo seemed just perfect, your experience is probably going to become worse.
I do think that is sad, and it’s important to own it, and apologize – I think having a place that felt safe and home and right become a place that feels alienating and wrong is in fact among the worst things that can happen to a person.
But the consequences of not making some major changes seem too great to ignore.
The previous iteration of LessWrong died. It depended on skilled writers continuously posting new content. It dried up as, one by one, as they decided LessWrong wasn’t best place for them to publish or brainstorm.
There’s a lot of reasons they made that choice. I don’t know that our current approach will solve the problem. But I strongly believe that to avoid the same fate for LessWrong 2.0, it will need to be structurally different in some ways.
An Atmosphere of Experimentation
We have some particular tools, and plans, to give authors the same control they’d have over a private blog, to reduce the reasons to move elsewhere. This may or may not help. But beneath the moderation tools and Public Archipelago concept is an underlying approach of experimentation.
At a high level, the LessWrong 2.0 team will be experimenting with the site design. We want this to percolate through the site – we want authors to be able to experiment with modalities of discussion. We want to provide useful, flexible tools to help them do so.
Eventually we’d like users to experiment both with their overall moderation policy and culture, as well as the norms for individual posts.
Experiments I’d personally like to see:
Bubbling Up and Peer Review
Experimentation doesn’t mean splintering, or that LessWrong won’t have a central ethos connecting it. The reason we’re allowing user moderation on Frontpage posts is that we want good ideas to bubble up to the top, and we don’t want it to feel like a punishment if a personal blogpost gets promoted to Frontpage or Curated. If an idea (or discussional experiment) is successful, we want people to see it, and build off it.
Still, what sort of experimentation and norms to expect will vary depending on how much exposure a given post has.
On personal blogposts, pretty much anything goes.
On Frontpage posts, we will want to have some kind of standard, which I’m not sure we can formally specify. We’re restricting moderation tools to users with high karma, so that only people who’ve already internalized what LessWrong is about have access to them. We want experimentation that productively explores rational-discussion-space. (If you’re going to ask people to only comment in haiku on a frontpage post, you should have a pretty good reason as to why you think this will foster intellectual progress).
If you’re deleting anyone who disagrees with you even slightly, or criticizing other users without letting them respond, we’ll be having a talk with you. We may remove your mod privileges or restrict them to your personal blogposts.
Curated posts will (as they already do) involve a lot of judgment calls on the sitewide moderation team.
At some point, we might explore some kind of formal peer review process, for ideas that seem important enough to include in the LessWrong canon. But exploring that in full is beyond the scope of this post.
Norms for this comment section
With this post, I’m kinda intentionally summoning a demon thread. That’s okay. This is the official “argue about the moderation overton window changing” discussion space.
Still, some types of arguing seem more productive than others. It’s especially important for this particular conversation to be maximally transparent, so I won’t be deleting anything except blatant trolling. Comments that are exceptionally hostile, I might comment-lock, but leave visible with an explicit reason why.
But, if you want your comments or concerns to be useful, some informal suggestions:
Failure modes to watch out for:
Types of comments I expect to be especially useful:
…
Ok. That was a bit of a journey. But I appreciate you bearing with me, and am looking forward to having a thorough discussion on this.