A few months ago we have launched an experimental website. In brief, our goal is to create a platform where unrestricted freedom of speech would be combined with high quality of discussion. The problem can be approached from two directions. One is to help users navigate through content and quickly locate the higher quality posts. Another, which is the topic of this article, is to help users improve the quality of their own posts by providing them with meaningful feedback.
One important consideration for those who want to write better comments is how much detail to leave out. Our statistical analysis shows that for many users there is a strong connection between the ratings and the size of their comments. For example, for Yvain (Scott Alexander) and Eliezer_Yudkowsky, the average number of upvotes grows almost linearly with increasing comment length.
This trend, however, does not apply to all posters. For example, for the group of top ten contributors (in the last 30 days) to LessWrong, the average number of upvotes increases only slightly with the length of the comment (see the graph below). For quite a few people the change even goes in the opposite direction – longer comments lead to lower ratings.
Naturally, even if your longer comments are rated higher than the short ones, this does not mean that inflating comments would always produce positive results. For most users (including popular writers, such as Yvain and Eliezer), the average number of downvotes increases with increasing comment length. The data also shows that long comments that get most upvotes are generally distinct from long comments that get most downvotes. In other words, long comments are fine as long as they are interesting, but they are penalized more when they are not.
The rating patterns vary significantly from person to person. For some posters, the average number of upvotes remains flat until the comment length reaches some threshold and then starts declining with increasing comment length. For others, the optimal comment length may be somewhere in the middle. (Users who have accounts on both Lesswrong and Omnilibrium can check the optimal length for their own comments on both websites by using this link.)
Obviously length is just one among many factors that affect comment quality and for most users it does not explain more than 20% of variation in their ratings. We have a few other ideas on how to provide people with meaningful feedback on both the style and the content of their posts. But before implementing them, we would like to get your opinions first. Would such feedback be actually useful to you?
Since I critiqued the graphs in the OP and offered to make them better, cleonid was gracious enough to provide me with the dataset for Eliezer (EY) and Yvain (Scott Alexander) (SA) posts and invite me to play with it. For convenience, I'll split my comment into two parts -- the preliminaries and the analysis itself.
First, two caveats.
I'm looking at data for posts by EY and SA. They, being superstars, are not representative of LW members. This analysis will not tell you, gentlereader, whether you would get more karma by making your posts longer or shorter (unless, of course, you are EY or SA :-D).
Next, the data that I have lacks timestamps. Therefore I'm forced to make the assumption (almost certainly false) that nothing changed with time and treat the data as a blended uniform mass. The temporal dimension is, sadly, lacking.
A brief description of the data: we have two tables (EY and SA) which list the number of characters in a post, as well as the number of upvotes and downvotes that the post received. There are about 3100 data points for EY and about 1400 for SA.
The distribution of post lengths for EY and for SA is rather different: EY posts tend to be much shorter. This is visible in the following graph which plots the empirical distributon of post lengths (cut off at 2000 chars, but there is a long thin tail beyond it).
In numbers, the median length of an EY post is only about 200 characters -- half of his posts are less than that. In fact, a quarter of his posts are less than 87 characters. On the other side, 10% of his posts are longer than 950 characters, 5% are longer than about 1500 characters. The mode of the distribution -- the most frequent length of the post -- is around 70 characters.
SA writes more: his median post length is 450 characters, more than double that of EY's. The shortest quarter of the posts is below 165 characters, the longest 10% are over 1900 characters and the longest 5% -- over 2700 characters. The mode for SA's posts is 180 characters, two and a half times as much as EY's mode.
Continued in post 2.