Of the title of this discussion post, we already have an approximation in the most voted-for lists for Main and Discussion. There are many problems with this metric, however, a subset of which are:
- Both sections are cluttered. The top comments for Main are full of rationality quotes (which are better accessed elsewhere), and the top comments for Discussion are full of polls. (Can we please have a non-stupid way of putting polls in comments?)
- Both sections are biased by exposure. A comment that a lot of people see generally gets more karma than comments which not many people see. As Less Wrong's growth rate increases, and as time goes by, this will increasingly bias these sections toward newer comments. Also, comments made by well-known LWers will be seen more often and correspondingly upvoted more.
- Joke comments oftentimes get a lot more comments than insightful comments.
- Entire threads can have weird voting patterns that don't match quality.
- These sections list number of upvotes, so extremely controversial comments appear in between unanimously good ones. (May not actually be a problem at all.)
- People use karma as behavior reinforcers, so comments like "I'll transcribe this" get lots of votes.
- Dozens more little things I won't try to list
- Don't post your own comments.
- Don't post more than one comment in this thread. If you find more other-person comments you want to add, edit yours to include them.
- Say why you think the comment is good, even if it's only a line.
- Bonus points for finding very old comments, or those from threads that got very little exposure.
- No joke comments or comments that solely quote people.
- Don't retrieve your comment from the list of top comments.
I like my idea. Let's see how well it works.
It also rules out how great this comment is (or could be, anyways).