In my experience, writing full-fledged, thoroughly researched material is pretty time-consuming, and if you push that out to the audience immediately, (1) you've sunk a lot of time and effort that the audience may not appreciate or care about, and (2) you might have too large an inferential gap with the audience for them to meaningfully engage.
The alternative I've been toying with is something like this: when I'm roughly halfway through an investigation, I publish a short post that describes my tentative conclusions, without fully rigorous backing, but with (a) clearly stated conclusions, and (b) enough citations and other signals that there's decent research backing my process. Then I ask people what they think of the thesis, which parts they are interested in, and what they are skeptical of. Then after I finish the rest of the investigation I push a polished writeup only for those parts (for the rest, it's just informal notes + general pointers).
For examples, see https://www.lesserwrong.com/posts/ghBZDavgywxXeqWSe/wikipedia-pageviews-still-in-decline and http://effective-altruism.com/ea/1f9/the_aidsmalaria_puzzle_bleg/ (both are just the first respective steps for their projects).
I feel like this both makes comments more valuable to me and gives more incentive to commenters to share their thoughts, but the jury is still out.
This is similar to the idea of an MVP in the startup world. It makes me think of a sentiment from Ryan Holiday's book on writing perrenial sellers: Ideas should become comments, comments should become conversations, conversations should become blog posts, blog posts should become books. Test your ideas at every stage to make sure you're writing something that will have an impact.
This feels like a classic Goodhart's Law problem. More comments are good for multiple reasons (feedback, positive reinforcement, more ideas, keeping up the conversation, and so on). It would be good to generate more comments, and do things that do this (as long as we avoid obvious provocations like #1 and #2 above), but if we start maximizing comments with our posts, then if you're not very careful about it, that path goes south quickly.
One thing I think is both 'safe' and effective over time is to reliably engage with your commentators. If you establish a norm that you're going to clearly read most or all the comments on your own posts, and reply to them when there's something to say (comments on comments are reinforcement the same way comments are) that sends a clear signal that comments are welcome and valued. I think that worked well on my own blog, but it's hard to disentangle with many things including 'I kept writing and got less awful at it in other ways' or with 'I kept writing and people finally started reading.'
I'd also note that from my Magic writing I developed the heuristic of a 1000:1 page view to comment ratio as the default outcome, so compared to that we're doing astoundingly well. My few examples of my blog being linked to by Scott and Marginal Revolution tell me that the 1000:1 remains about right in those contexts.
What about asking your audience questions?
For example, you could ask questions:
* Seeking criticism, such as "I think section x is the weakest part, what are some alternative arguments?"
* Promoting understanding, such as "Can you think of 2 more examples of <concept I just introduced>?"
* Stimulating research, such as "I think this model can be applied to area y, does anyone have suggestions for how to do this?"
This might help get readers out of passive consumption mode, and into thinking about something they could comment about. It would also make the writing more useful.
The situation you find yourself in is called Warnock's Dilemma, named after Bryan Warnock, one of the early Usenet pioneers. He said that silence can be interpreted in one of five different ways:
The post is obviously correct, and so well researched that nothing more needs to be said.
The post is complete and utter nonsense but no one wants to waste the time or energy to point it out.
No one read the post.
No one understood the post and thought it would be worth their time to ask for further clarification.
No one cares about the post.
After reading Ryan Holiday's book, Trust Me I'm Lying, I'm not sure that it is possible to incentivize comments without falling into the Buzzfeed trap of writing content with high emotional valence in order to drive engagement. I think that the way to improve one's writing is to seek out people and ask them to give comments (and incentivize this, either through social means or even via financial means, if you can afford it). Relying on one's writing alone to draw commenters sets up an incentive gradient that leads straight to Buzzfeed, and I'm not sure that there's really any way out of it.
After reading this, I tried to figure out what the desired end goals were. I think we can all agree that commenting for the sake of commenting is meaningless, so I looked for what would fill in the blank "commenting for the sake of ______".
At the beginning of the post I saw:
1) I like getting comments (I see this as a bid for attention)
2) Staying in the public view (more comments = more staying power)
3) Learning on the side of the reader (commenting = engagement = better learning)
At the end of the post I saw:
4) Public attention incentivizes efforts to research and learn (I see this as attention as well)
5) People writing for comments write more engagingly
6) Learn how to comment better (what's the goal of a comment? To learn better?)
7) Engaging writing decreases the chance that good thoughts don't get lost (public view again)
With that, I'm seeing two main ideas: bids for attention and learning on behalf of the public. Bids for attention aren't bad and I think a community has to respond to them for the community to work well. If a community never tells its members that they are seen and noticed, the community isn't going to last very long. How does it change the writing and responding if we acknowledge these bids?
On public learning, I think it might help to look into the research on teaching. Additionally, the assertion of teacher and student will have to be addressed. The approach of "I have things you need to learn" is a bit off putting and likely to turn people away. However, this community seems to want to learn, so maybe a call for teachers on topics the community is interested in, and that can establish who is teacher and who is student.
On specifically the teaching/learning part, research in teaching shows that lecturing is very ineffective for learning. Many times, long written posts are simply data dumps that are similar to lectures. Effective teaching puts effort on the learner, tolerates and corrects mistakes, and encourages trying. The teacher may have the knowledge but doesn't give the knowledge and instead helps the learner find the knowledge. I have no idea how well this would work in a web forum.
I think there are more ways that writing and commenting can be used, but these were the themes I picked up from this post. Perhaps by bringing them forward like this, the other uses of writing can be drawn out.
Upvoted for spelling out thinking in a goal-oriented way that functions a) as some useful insights, and b) a useful example of how other people could approach writing similar comments.
Be Wrong - It has been said: if google fails you, the fastest way to get a question answered is to post a wrong answer on reddit. This will result in a lot of flood of people explaining things to you.
It strikes me, in light of Scott Alexander's recent post on predictive processing, that this might not be quite as "evil" as it seems. Maybe seeing a wrong answer triggers a prediction error which brings more relevant information into conscious awareness, actually making the original question easier to think about.
Huh, that may be.
Worth noting: the strategy I end up using (post things that aren't fully thought out, label them as-such, and then I get to be wrong about some things without doing it *quite* on purpose), is something I think I mostly endorse, it's just that I don't want it to be the only kind of post that gets a lot of comments.
Scott Aaronson's gone on record as saying that he gets a lot of his research ideas from engaging with people he disagrees with and trying to explain carefully what his disagreement with, or something like that. I think there's some interesting room for periodically attempting to be Wrong on Purpose to see what reactions it inspires in other people.
What if there were different sections for different types of comments? Ex. a section for:
"I agree"-type comments
"Stupid question"-type comments
Very low barrier "open thread/random" comments
Typical high barrier "meaty" comments
Right now, I suspect that people with "I agree", "Stupid question" or "open thread/random" comments don't post them because they're "not good enough". But if there was a section explicitly asking for these types of comments, I suspect that people would be very willing to provide them.
Another benefit is that comment readers could skim for the types of comments they're interested in reading.
I'm not sure if the sections I proposed are the best ones to have, just thought a concrete example would make things easier to describe.
I don't think the UI for comment sections would make things overly complicated. Something like Bootstrap's nav tabs should do.
Something similar, previously suggested in the meta-section was to allow for "short comments" that show up collapsed by default for everyone but the author, which'd include "thanks, I agree!" and "I noticed a typo" type comments.
I'd be worried about creating a distinction between stupid questions and "meaty" comments, since then people will be worried about "is my question meaty or stupid"?
I'd be worried about creating a distinction between stupid questions and "meaty" comments, since then people will be worried about "is my question meaty or stupid"?
Good point, I agree.
Writing this comment to provide social validation!
Like you, I have this norm in my head that I should only comment if I have something concrete to add, rather than to just encourage or discourage. But I wonder if this comes from a different context? I don't remember how I learned this norm, but many places on the Internet get enough posts and comments at a low enough average quality that I don't really value having another marginal post or comment unless it's raising the average quality. By contrast, the marginal post or comment on LW is interesting to me even if it's not raising the average quality, because the average quality is pretty high and I literally run out of posts and comments to read. So maybe the norm is not serving its original purpose.
As a blogger, I found (a few years ago) I reliably got more comments on what I thought were my *worst* posts. Now that's more or less no longer true; I can write stuff that's reasonably "meaty" for a blog post and still spark debate, though I'll never get a ton of attention as long as I stay out of the Petty Bullshit Gravity Well. Full-fledged theories are bad for getting comments (and also more likely to be wrong in retrospect). Semi-formed ideas fleshed out with facts and references and imagery are better.
That's interesting.
Semi-formed ideas fleshed out with facts and references and imagery are better.
Was this the main thing you do now that you didn't do a few years ago? Anything else you can think of that's changed? (It does very much match my experience of the ingredient))
Full-fledged theories are bad for getting comments (and also more likely to be wrong in retrospect).
Interesting - this seems obviously usually-true in retrospect but I hadn't thought of these two facts at the same time. I think the things separating a full-fledged theory (that's wrong) from a something-less-than-a-full-fledged-wrong-theory, is something like:
Having had a week to think even more about this (after deliberately not having a dedicated conclusion when I posted this, for similar reasons), I think my thesis statement for "what good kinds of posts will get the attention and comments that we want", is something like:
Be slightly wrong, and leave people with unsolved problems, with "Semi-formed ideas fleshed out with facts and references and imagery" being a pretty good way to accomplish that.
The main difference between my blogging "back then" and now is that back then my goal was to write things that were absolutely true and absolutely uncontroversial. I didn't achieve this goal -- it was never realistic for the blogging medium. And sometimes I threw caution to the wind and wrote extremely controversial things. I'm now aiming for pretty true and not very controversial.
You mention wanting to be incentivized to research things, and also that a particular danger to the community is writers optimizing for engagement at the expense of other things.
It seems like a possible partial remedy for this would be a mechanism for the readership to make their desires known in a centralized place. Right now, a hypothetical writer William, if they want to craft content the community wants, would be best served by doing a review of past posts in search of things which are consistently popular. If they are lucky and clever, they may even be able to infer a niche which hasn't been filled - a gap in some topic, or perhaps a missing topic altogether - that would be welcome. This is time-consuming and presumably error-prone. Perhaps more importantly, it seems likely to produce content which is similar to what has come before. (Disclaimer: I have never made a top-level post, so if there is a different, quicker, more accurate process, please let me know.)
If instead the community could vote to say "We would like more posts on Machine Learning" or "We want posts on what Dr. X's latest psychology research means about consciousness", this would create an easily visible incentive to research and an incentive to explore specific topics (which may steer writers away from optimizing for comments, although the comments will likely follow). It adds complexity to the site, but I think it may be worth an experiment.
A visible community vote on what we would like to see more of might get a flurry of people to cross the lurker/poster divide, but when I picture myself seeing that feature I wind up thinking about it as a way I could ask ("vote") for something I wanted the particularly prolific or especially insightful writers here to talk about. Imagine ten thousand lurkers voting for a Project Hufflepuff update or something like that. A variation that might incentivize new writers might be an open vote, with what kind of posts each user wants placed somewhere notable on their profile. I ran across TheZvi's You're Good Enough post a few weeks ago, and that's likely to be the kick I needed to start blogging long form again. (Status: Two half-written ~3k word posts. If I don't have a blog I can point people at by the 15th of October, I will admit that it wasn't enough of a kick.)
On that post, TheZvi offered to read things that were written because of their post, and knowing I have at least one guaranteed reader makes writing a lot easier for me. Commenters often want engagement of some sort. Stating at the end of a post that you precommit to reading the first ten on-topic comments and responding to them all with a comment of your own or the first ten blog posts responding to your own post that people link you and commenting on those posts seems like it might incentivize people to write the kind of responses you want them to write. This would allow the people who write a lot and who large parts of the community read to steer the conversation more; whether this is good or bad is a reasonable question but the notion isn't obviously terrible. It would at least create a variety of topics budding writers could be directed at; a single outcome from a LessWrong topic vote might result in a dozen new posts about that one topic, while a different "this is the kind of content I want" statement from the widely read authors would be more likely to result in three or four posts on each of their topics.
I don't think this would require any technical changes, and I think it would also be a worthwhile experiment.
Interesting idea, I like it! Seems like something that could be tested by just having a Request for Articles post, and have people post + upvote requests via comments.
Personally, I'd like to see a post on Cognitive Behavioral Therapy.
Venue also matters a lot through the social context it brings. Individual Wordpress blogs often feel like you're saying "this is where my writing lives; by commenting, you're coming into my house", which can be challenging to take lightly -- especially when you're talking about a neighborhood of individual blogs, few of which get regular comments. Meanwhile social media is a weird mix of jokes and personal content with discussion-oriented ideas, where there's an uncertain rudeness in potentially burying someone with attention or notifications by Starting Discourse. And in both of these, if it's not controversy or gossip or dilettantism, then posting the most makes you king.
So I was/am hopeful about posting more to LW 2.0 largely for the sake of better defaults around "this is for having a conversation" -- both "formally" in responding directly to or building on the OP, and more "socially" or indirectly by contributing thoughts on the same subject, and in a venue with moderation and karma where things can bubble up without the speculative/social/everyone's-an-expert elements (or sheer consistent quantity).
I find that my writing seems to actively repel comments compared to stuff that gets comparably received by other metrics. I do try to go out of my way to write mostly on the rare occasions I have something unambiguously sensible or useful to contribute; it earns me a high upvote/downvote ratio, but little sense of how people are engaging with what I have to say.
At the same time, maybe this makes me part of the problem of silence on the best writing. I'm also interested in learning to be a better commenter, but I'm not someone who thinks they can or should always have something to say. For my part, I think this mostly indicates that I should comment more with thoughtful questions, but I'm very interested in you or anyone else fleshing out your "being a better commenter" open problem -- I think this is potentially more important for success here than writing the right kinds of posts.
I wonder if it would be useful to have some kind of informal peer review for boring important posts.
My solution is that I'm motivated by the process of writing, and so if anything am less likely to write high-comment-count articles, because it's stressful when lots of people disagree with me. (Although I end up writing a lot of social reality posts anyway because that is what I have opinions about.)
In fandom, people tend to leave comments that talk about particular emotions they had or lines they thought were particularly funny or insightful, so you get a lot of comments on widely read things even if there is nothing wrong with them. I find myself reluctant to leave such comments on lesswrong 2.0 because I don't want to clutter everyone's recent-comments feed.
Promoted to featured for writing up thoughts from over a long period, about an important problem for the community, and provoking great, helpful comments.
1. Be Wrong - It has been said: if google fails you, the fastest way to get a question answered is to post a wrong answer on reddit. This will result in a lot of flood of people explaining things to you.
Reminds me of the old joke: if you want to find out how to do X in Linux, asking about it is useless. But going to the #linux IRC channel to say "Linux sucks because it doesn't even let me do X", will give you the answer of how to do X in no time.
I am working on plans for a sequence that I think has a shot at being 6, though I worry about my writing quality. (edit clarification: I will probably be ready sometime early next year. I've been pondering how to do this for a while, and I'm starting practicing writing skill I'll need.)
This post has a good quality of meta. The advice mostly applies internally.
It would be nice if there was a magic number in the font attributes, supplementing size/color/etc, which modulated people's internal quality bar, ranging from "facebook" to "academic journal". A website could dynamically set this based on the difference between the ideal amount of engagement and the amount observed.
This seems to be a problem which crops up again and again in different forms: a discussion/location/thing gets good, and implicitly, the quality bar rises. Maybe it is because an audience showed up, or maybe it's just because you start worrying about ruining the quality of what has gone on so far. This takes away the original sense of freedom which often produced the quality in the first place. You start a tumblr to put writing that's not good enough for the blog. Your Twitter account gets popular and you retreat to an alt account, only to find that it gets popular, making you retreat to another alt... That's a bit different from what you're pointing at, but it seems related.
Realistically, how does one hang a little sign that says "community space, feel welcome to engage"? I suppose a sense that people know each other is important to this. It's part of the reason Facebook feels easy; you are talking to your friends.
> Realistically, how does one hang a little sign that says "community space, feel welcome to engage"
I think literally just saying that at the beginning (or end) is actually pretty decent. (See my opening note for Musings on Doublecrux, which [I think not-entirely-coincidentally] got 70 comments)
LW2.0 has vague plans for a "shortform/informal" section, which might help address the issue, but honestly even if that worked it'd still leave the problem for Posts on the Front Page, where I'd like more comments to be regardless.
[Edit] Musing....
Something that occurs to me re: comment quality, though, is that if the site scales, it's *fine* to raise the bar on what makes a "good enough comment." For example, Slatestarcodex gets hundreds of comments each post. I wish it got less, since that's too many to skim through. Slatestar posts on LW get... 3? (in part but I don't think entirely because people comment on the original)
I'd prefer if LW-crossposts of Scott Alexander essays ended up in the 20-50 comment range, and stayed there, and as the number of people following on LW increased, people's intuitive threshold for what's-worth-posting increased so that we kept the 20-50 range.
I suppose the size of the audience is very important here. If there is a large enough audience then good posts will still receive a reasonable number of comments even if they get less than posts which have more of the factors that you described. On the other hand, when the size of the audience falls below a critical mass this kicks off a Negative Feedback Loop where less commenters means less posters and hence less commenters and so on and so forth. I'll be curious to see whether LW2.0 had gotten us above this critical mass.
I wonder how much time one needs to recognize a higher-value comment irl. I mean, we could try each choose a post and come to the meetup to talk specifically about them. (Maybe not warn one another what it would be.) And then post the distillated hive-comments as responses... ...I also find it useful, but hard, to exit 'editorial mode' when commenting. Editorial-like comments are quick and mostly shallower than epistolary-like ones. It helps to imagine that you are actually mailing a letter to someone you give a damn about (and Lewis Carroll, of course, has advice on how to answer letters - https://en.m.wikipedia.org/wiki/Eight_or_Nine_Wise_Words_about_Letter-Writing )
There are a whole lot of social networks out there, each with its own rules and karma system. I wonder if anyone has done a survey of what works and what doesn't?
More specifically: for the past few years I've had an account at rpg.stackexchange.com. I've found that this site performs its function remarkably well. When I read questions on this site, I feel motivated to write clear, helpful, somewhat-researched answers.
There are lots of other social networks. Reddit comes immediately to mind. I think slashdot has put a lot of effort into their system. And of course there's facebook and twitter and tumblr. I don't know much about these sites, except that I find them sort of irritating, particularly in the comments section. But probably someone who uses those sites more would have more to say.
If we're really dedicated to figuring out how to do comments properly, it seems like we should start by doing a survey of what already exists.
I've definitely done a lot of personal case-studies of all the major platforms, but haven't gotten around to writing up my thoughts yet.
I've also been reading through this book, which is quite good and analyzes quite a few existing systems and paradigms: https://smile.amazon.com/Building-Successful-Online-Communities-Evidence-Based/dp/0262016575?sa-no-redirect=1
Maybe part of the problem is that some topics lend themselves to short-form discussion, while other topics lend themselves to long-form discussion? There's lots of great posts I read where I don't feel the urge to comment, but do feel the urge to research a little and then write a long-form response - heck, that's how I feel about probably a third the stuff Scott writes. But the current standard interface encourages a long-form post followed by short-form comments.
Maybe one solution would be encourage "comments" which are links to a long-form response, along with a short summary? Maybe make such responses more discoverable in comment feeds?
Epistemic Effort: Thought about it for a year. Solicited feedback. Checked my last few posts' comment count to make sure I wasn't *obviously* wrong.
A thing that happens to me, and perhaps to you:
Someone writes a beautiful essay that I agree with, that sheds new light on something important.
I don't have anything really to say about it. I don't want to just say "I agree!". So instead of commenting, I give it an upvote and move on.
This feels bad for a few reasons:
I've been impressed with how much quality writing has been going on on LW2.0 so far. There has been some but not as much commenting as I'd like.
I've gotten a sense of what inspires interesting, meaty discussion.
Unfortunately, most of it seems... kinda bad?
Things That Get People To Comment
1. Be Wrong - It has been said: if google fails you, the fastest way to get a question answered is to post a wrong answer on reddit. This will result in a lot of flood of people explaining things to you.
2. Be Controversial - Even better, post something that some people think are wrong. Then you get a bunch of people commenting to correct you, and then other people who disagree correcting them! The arguments perpetuate themselves from there. You won't even have to do any commenting work yourself to keep it going!
[BTW, these are observations, not recommendations. This list is optimized to answer the question "what causes comments" not "how to make the world better."]
3. Write About Things People Feel Qualified to Have Opinions On - If you write a post on machine learning, and post it somewhere where nobody really understands machine learning, it doesn't matter if you're wrong or controversial! Nobody will understand enough to care, or feel confident enough to argue. Some considerations:
4. Invoke Social Reality - People pay more attention when you're talking about social norms, or about changing coalitions of people, or arguing that some people are Bad and Wrong. This is for two reasons:
5. Be So Inspiring That People Create Entire Fandoms of Your Work - This worked for Eliezer and arguably Scott. It can probably be broken down into smaller steps. It's pretty hard though. And a bunch of people trying but failing to do this can be annoying. (I've tried/failed to do this sometimes)
...
And then there's...
6. Leave People With An Unsolved Problem That They Care About - This is related to "they feel qualified to have opinions", with the followup step of "there is actual useful thinking they can contribute to, either to solve your problem, or to apply your idea to solve their problems."
Things I've Noticed Myself Doing
Since comments are socially validating, I've noticed a tendency for me to end up writing:
This doesn't all seem bad, necessarily. But I've noticed other people that seem to be doing similar things. I've also noticed some people who tried to get people to talk about important things, and failed, and gradually resorted to writing more provocative things to get people to pay attention (which succeeded!).
It seems like a rationality community warped by those incentives isn't going to accomplish the things it needs to.
So, some open problems I'm thinking about, which maybe are relevant to you:
That's all I got for now.