All of owencb's Comments + Replies

Thanks, I think that these are good points and worth mentioning. I particularly like the boundary you're trying to identify between where these decentralized mechanisms have a good track record and where they don't. On that note I think that although academia does have complaints about political bias, at least some disciplines seem to be doing a fairly good job of truth-tracking on complex topics. I'll probably think more about this angle.

(I still literally agree with the quoted content, and think that decentralized systems have something going for them which is worth further exploration, but the implicature may be too strong -- in particular the two instances of "might" are doing a lot of work.)

I think keeping some dependence on quantity is desirable, but that scaling linearly with number of posts weights it too heavily compared to variation in number of upvotes (I proposed scaling with roughly the cube root of number of posts in my explicit formula suggestion elsewhere in the comment thread).

A good system for finding old content. Often old posts or comment threads represent the best content on a given topic, but if there isn't a way to browse by topic they're basically lost. This makes me less excited about engaging in the first place -- vice versa a sense that something will persist is a draw in a venue for circulating ideas.

This is one thing where I think the forums/subforums structure of old UBB boards is actually superior to most more modern replacements (although such boards have other significant problems, most notably lack of threading).

2habryka
Interesting. I agree with this, but am not super sure about the best solution. I was hoping we could fix this with something like the sequences feature, which would be optimized for creating persistent content, and a sense of canon in the community. I.e. there would be a set of sequences that would be considered core, and that would be prominently placed on the page. Users would have the ability to add sequences, and if they are good enough and popular enough, they get added to the core sequences This seems to make past content more accessible, though it is better suited for making top-level posts findable, and worse suited for making very good discussions findable. Unsure how to best make the discussions more findable.

Here's an example functional form which is the best guess from the top of my head at creating this effect (but I'm giving as an illustration of what to pay attention to rather than a claim that this precisely should be used):

K = (U - 3D) * P^0.3 / R

Where
K = Karma
U = total (weighted) upvotes
D = total (weighted) downvotes
P = total number of posts+comments
R = total number of reads of your posts+comments

7Wei Dai
I also agree with the spirit of this, but I think dividing upvotes by the number of reads penalizes reads excessively, because each reader doesn't decide how to vote independently. Once a post already has a high score, a new reader is not likely to upvote it more even if they think it's high quality. Also we ought to encourage people to create highly popular articles that spread our ideas beyond the local community, and this system would serve to discourage that. On the other hand we also don't want to penalize people for writing specialized content that only a few others might read. I'm not sure what the right solution is here.
4Marcello
I agree with the spirit of this. That said, if the goal is to calculate a Karma score which fails to be fooled by a user posting a large amount of low quality content, it might be better to do something roughly: sum((P*x if x < 0 else max(0, x-T)) for x in post_and_comment_scores). Only comments that hit a certain bar should count at all. Here P is the penalty multiplier for creating bad content, and T is the threshold a comment score needs to meet to begin counting as good content. Of course, I also agree that it's probably worth weighting upvotes and downvotes separately and normalizing by reads to calculate these per-(comment or post) scores.
1habryka
I was just writing a very similar function in one of the comments above! I think something in this direction makes sense.

One variation of karma system I'd like is the ability to rate posts as being exceptionally good (probably taking more than one click, to introduce a trivial inconvenience so that it isn't used all the time like five star ratings are). This would give more ability to pick out very useful contributors from small numbers of posts.

7habryka
Agree. One of the broader things I have been thinking of is a similar two-tier voting system like Facebook has. There is the primary interaction of upvoting and downvoting, but then there are additional vote-types you can access with an additional click (on Facebook "angry", "sad", etc. here it would be "exceptionally good point", "needs clarification", "too agressive" or something along those lines).

You talk about using karma thresholds for various things. But traditional lesswrong style karma screens more for quantity than quality of posts, and this would remain true of a version where you weight people's upvotes and downvotes. I suggest looking for versions which filter more for quality (while not creating too much disincentive to make additional posts/comments).

5jimrandomh
This is actually a feature, not a bug. The karma threshold isn't just there to limit who has access to features; it's also to increase the cost of creating sockpuppets and of recovering from bans.
3habryka
My current interpretation is that you mean that people who write a lot of content generally get much more karma than people who write little but very good content. I agree with that, and have been thinking about good ways of dealing with that. Here are two approaches: * When deciding what to show the user, use an algorithm that combines the information of: 1) How many upvotes did this piece of content get? 2) How many users have seen this piece of content? 3) How many downvotes did this piece of content get * Allow users to give out variable Karma rewards, with some cost attached to them. Maybe Karma transfers, or some limited amount of currency that's generated based on your current karma. Top comments would then receive more of this limited amount of currency.
4owencb
Here's an example functional form which is the best guess from the top of my head at creating this effect (but I'm giving as an illustration of what to pay attention to rather than a claim that this precisely should be used): K = (U - 3D) * P^0.3 / R Where K = Karma U = total (weighted) upvotes D = total (weighted) downvotes P = total number of posts+comments R = total number of reads of your posts+comments
3owencb
One variation of karma system I'd like is the ability to rate posts as being exceptionally good (probably taking more than one click, to introduce a trivial inconvenience so that it isn't used all the time like five star ratings are). This would give more ability to pick out very useful contributors from small numbers of posts.

I'm in favour of this, but I think it would be even better to give high weights by fiat to some trusted users, in a way that grounds the whole Karma system in something, and makes it harder for low-quality bubbles to self-perpetuate.

2habryka
The current plan is to seed the karma system with a set of highly trusted users, and then do something similar to Eigenkarma. Which would both ground the system in something, and allow for dynamic karma allocation.

I think keeping track of which posts you've read would be useful for all posts, not just sequences.