So I did what you suggested and plotted the number of top level posts and total posts over time. The attached graph is averaged over the last 20 open threads. Code available here: https://gist.github.com/TRManderson/6849ab558d18906ede40
I don't trust myself to do any analysis, so I delegate that task to you lot.
EDIT: Changed GitHub repo to a gist
For those who don't want to guess or dig into the source, the missing x axis unit is "minutes".
That's probably when people are like "I will rather wait for another open thread, so my comment has more visibility". If we will have open threads more frequently, that moment might just come sooner.
However, I am in favor of an experiment.
Posting near the end of an open thread's lifetime is (maybe) like cooperating near the end of an iterated prisoners' dilemma: others benefit (by having interesting discussions available more of the time) but you lose (by having what you post seen by fewer people).
One way one avoids this in IPD tournaments is by making the end uncertain (e.g., each turn after the 100th there's a 1% chance that the game ends). It would be interesting (though, frankly, probably not worth the effort) to randomize the starting of open threads in a similar way: e.g., once 3 days have elapsed, each hour a new thread gets started with probability 1%.
Could you add labels so stupid people like me could figure out what that graph means? I'm guessing the blue line is top level comments and the green line is total comments; both averaged from all the weekly open threads. The y axis would be number of posts and the x axis should be time but I'm not sure what unit it's in. Also, why does the blue line suddenly stop?
The blue line suddenly stops because the last comment is posted at that time. I was kind of lazy about this graph and did have labels and a legend, but apparently I was too out of it to realise they didn't show on the png.
As said by gwillen, x axis is minutes.
Sorry about not having units, I added code to set them but apparently it was the wrong code and I wasn't paying enough attention.
Green line is total comments, blue is top level comments. X-axis is minutes, y axis is number of comments.
An automated daily thread for perishable issues would probably be the way to go. I have seen it work elsewhere. The implementation would be as simple as a crontab entry on someone's Linux box and a script to make a post.
Just an idea: We could post them as needed based on the number of comments in the current thread. I would think ~200 would be a good threshold. We could always lower or raise the threshold later.
ETA: I see that this could be a problem for people if the thread got to say 190 and no one wanted to post for fear of getting drowned out. However, we could have a deadline for reaching the threshold, which if exceeded, would simply start a new thread. For example, if we had a deadline of one week, it would be no worse than the current OT situation, but it might be better if the threshold were crossed earlier.
I am against more frequent threads, as it'd likely make things harder to find and the individual discussions shorter and shallower.
What's the big problem with people waiting a few days to post when they want full exposure, anyway? It is a useful schelling point, and it ensures that those comments will get said exposure for a full week.
I feel that its only worth posting something to the open thread within the first 2 days after it is opened. Anything after that gets buried.
Many people use open threads to share links to various articles. Perhaps it is media threads that should be more frequent? E.g. Maybe we should have Written Media Threads and Visual Media Threads? Or perhaps open threads should have topics? For example, Open Thread for asking for advice, Science Open Thread, Social Science Open Thread, Futurism and Transhumanism Open Thread, Artificial Intelligence Open Thread, Open Thread to share links to various blog posts, long form articles, Open Thread that is similar to Reddit's Best Of (people would link to insightful comments from outside of LessWrong), etc. Some special threads, e.g. Group Rationality Diary are already like specialized open threads.
Having open threads every day would clutter Discussion and it would make finding anything very hard. "Open Thread" is a very non-informative title.
I think that one of the reasons why too few people comment on old threads is that LessWrong post sorting algorithm, being based on Reddit, doesn't move the threads that have recent comments to the top. At the moment the only way to notice that the old thread is now active (unless someone replied to your comment) is to notice its activity in "Recent Comments" section.
Interesting. I like the idea to have specialized OTs for different subjects. We could combine this with zedzed's suggestion and have a sub for "Open Threads" or something similar. Then we could omit the redundant "Open Thread" from the post titles, and just have them say the subject and the date.
No. As long as the open threads keep expiring and being replaced they will 'die' some time before their expiration date. Having them last a whole week leaves at least a couple of days in which discussions can be started, and then a few more days for each subthread to play out before everyone moves on.
Personally I fear that my contribution will drown unless posted early on so I tend to wait if I want to add a new top level post
If you think your contribution is important enough that you don't want it to drown, how about simply opening a new post in discussion?
If one operates on the goal of "I want somebody to see this, ideally soon, but I don't want it occupying a whole slot of the discussion list for ", then it seems like a reasonable idea to use an open thread which is not nearing expiration. A lot of the stuff posted in OTs is basically personal - stuff that other people are much less likely to care about responses to than you are - and making a new post (even in discussion) feels spammy for that in a way that posting in a OT doesn't.
I personally like the suggestion of having a section just for open threads, where things that people would post as a top-level comment in any given OT can instead be made a top-level post there. It would iterate quickly, of course, but that's not necessarily bad; every top-level would have some time in the limelight.
Alternatively, is there a reason the OTs need to expire? Obviously they run the risk of getting extremely long to load if there are too many comments, but surely that's a problem the codebase can cope with. It already highlights new (posted since your last visit to a thread) comments, including new top-level ones. Is there a way to skip to the first/next new comment in a thread?
Meta question, on the other hand: is the issue of these cyclic waves of OT activity sufficiently problematic to warrant significant changes? (My feeling is yes, but mostly because the cost seems minimal; I guess I don't feel like the issue itself is that severe.)
The open threads have always seemed terribly inefficient to me. Most forums have a board for "stuff that doesn't belong anywhere else." That seems to be the purpose that the OT is being used for, but it's not terribly effective at it.
Any topic-thread that's posted regularly should really be a subreddit, IMO.
Absolutely. The proper response to this confusion should be: "fix the site to have a third, lower priority level", not "increase the frequency of our hack".
Yeah, it fills up to fast - it starts feeling uncomfortably full after just one or two days for me.
So having it every other day might be worth a try. I guess the main issue is that if we have them too often, it will clutter up /r/discussion.
Given how much activity happens in open threads (glancing through /r/discussion, it looks like 1/2 to 1/3 of the comments), maybe have open threads every three days (days divisible by 3, perhaps) and give them their own sub (so, Main, Discussion, OpenThread).
Prediction: 10% chance we increase frequency of open threads, 20% they get their own sub if we increase frequency. I haven't observed LW changing much, and the largest changes to the site since I arrived was instituting a new moderator.
I started reading this with the impression that open threads should be more frequent, but I changed my mind as I've read the previous comments. I'm not calling for open threads to be posted more often. However, I too support an experiment.
I want to point out that similar to having open threads with specific topics, users can also post or share a periodic thread that hasn't been around for a while. I've noticed user Gondolinian has been doing this with diligence recently. He has posted the latest welcome thread, Lifehack Ideas thread, and Stupid Questions thread. So far as I can tell, the only protocol for inaugurating a new periodic thread is by posting it, trying it out, and seeing if it's well-received. That risks one's own karma, but I believe the blow could be softened by including in the original post a caveat about how this thread is being tried out for the first time, and it will not be continued if nobody appreciates it. If anyone reading this is a regular reader of the open threads, and you notice a pattern of the same topic coming up time and again, and being very popular, maybe try starting a periodic thread on the subject. The outcome of all of this might mean there are more specialized threads which make searching for content easier, don't limit community feedback, and hopefully lessen the traffic on the generic weekly open threads.
This is a tradeoff of higher complexity vs added benefits. From TRManderson we see that weekly open threads are well used throghout the week and even after (!), so the benefit would probably not be that people would post more.
What could be another benefit of having a higher frequency?
It's also possible some of the open thread comments should have their own discussion thread. Maybe there should be a separate open discussion area that sorts the open thread discussion better. Main is basically just featured posts and Discussion is really the meat of the site.
Currently open threads are weekly and very well received. However they tend to fill up quickly. Personally I fear that my contribution will drown unless posted early on so I tend to wait if I want to add a new top level post. Does anyone else have this impression? Someone with better coding skills than me could put this statistically by plotting the number of top level posts and total posts over time: If the curve is convex people tend to delay their posts.
So should open threads be more frequent and if so what frequency?