So here's a thing I'm a bit uneasy about with this, after living with it for a while: since every comment is, by default, strong-upvoted by its poster but only weak-{up/down}voted by others, and since most comments don't attract huge numbers of votes, it seems as if a typical comment's score says much more about who posted it than about what anyone else thought of it, and I'm not sure that's how we want it.
Is it feasible to (1) make comments be only weak-upvoted by their posters, and (2) remove your ability to vote on your own comments?
(I don't know whether that's the right solution. Suppose A posts something and B really dislikes it and strong-downvotes it, and no one else cares much. Then the comment's score will be negative even if A has much more karma than B, which may not be what we want.)
Update: This has now been pushed to the live site. For now, on Desktop, strong upvotes require click-and-hold. Mobile users just tap multiple times. This is most likely a temporary solution as we get some feedback about how the respective modes work under realistic conditions.
It's likely there'll be a few kinks to work out. Let us know if issues come up.
We got to discussing this on #lesswrong recently. I don't see anyone here pointing this out yet directly, so:
Can you technically Strong Upvote everything? Well, we can’t stop you. But we’re hoping a combination of mostly-good-faith + trivial inconveniences will result in people using Strong Upvotes when they feel it’s actually important.
This approach, hoping that good faith will prevent people from using Strong votes "too much", is a good example of an Asshole Filter (linkposted on LW last year). You've set some (unclear) boundaries, then due to not enforcing them, reward those who violate them with increased control over the site conversation. Chris_Leong gestures towards this without directly naming it in a sibling comment.
In my opinion “maybe put limits on strong upvotes if this seems to be a problem” is not the correct response to this problem, nor would be banning or otherwise 'disciplining' users who use strong votes "too much". The correct response is to remove the asshole filter by altering the incentives to match what you want to happen. Options include:
Note: I would never punish anyone for their vote-actions on the site, both because I agree that you should not punish people for giving them options without communicating any downside, but more importantly, because I think it is really important that votes form an independent assessment for which people do not feel like they have to justify themselves. Any punishment of voting would include some kind of public discussion of vote-patterns, which is definitely off-limits for us, and something we are very very very hesitant to do. (This seemed important to say, since I think independence of voting is quite important for the site integrity)
(Note: still disenfranchises users who don’t notice that this feature exists, but maybe that’s ok.)
It is not difficult to make people notice the feature exists; cf. the GreaterWrong implementation. (Some people will, of course, still fail to notice it, somehow. There are limits to how much obliviousness can be countered via reasonable UX design decisions.)
This is also a UX issue. Forcing users to navigate an unclear ethical question and prisoner’s dilemma---how much strong voting is “too much”---in order to use the site is unpleasant and a bad user experience. There should not be a “wrong” action available in the user interface.
[emphasis mine]
This is a good point, but a subtle and easily-mistakable one.
There is a misinterpretation of the bolded claim, which goes like this:
The UI should not permit an action which the user would not want to take.
The response to this, of course, is that the designers of the UI do not necessarily know in advance what actions the user does or does not want to take. Therefore let the UI permit all manner of actions; let the user decide what he wishes to do.
But that is not what (I am fairly sure) nshepperd meant. Rather, the right interpretation is
...Some important background thoughts here:
I think a lot of reaction against wide-ranging-upvotes is that upvote/downvote used to mean something clear and legible, and (both with the original advent of LW2.0, and more so here), it's turning to something less legible.
And I think, importantly, the legible version of it was wrong – the sort of thing that's easy to measure but not the actual thing I care about.
So one option is to make peace with the fact that karma-scores don't refer to a clearly legible thing.
Another thing, which might be correct, is to make karma even less legible – rather than have it look superficially like other karma systems that refers to a sort-of-legible thing, instead karma is just, like, a percentage-score "how good is this post probably?" or, hell, like, a color, or doesn't appear at all or something.
(Or a combination, where for the sake of transparency the meta-data of what's going on under the hood is showed on hoverover, but the top-level result of karma is a color/percentage/vague-thing-of-some-sort)
I want to suggest having new posts/comments be normal upvoted by default (by the author) instead of being strong upvoted, for a couple of reasons.
(I think I might actually prefer to go back to the LW1 system we had at the end, but I'm not sure if that's a rational response or if I'm just used to it and don't like change anymore.)
(the "initially high karma will make people less likely to upvote things" is an interesting concern that hadn't occurred to me. I'm not sure how to best test it, but thanks for bringing it up as a thing to watch out for)
Another issue that I don't know if you've thought of: strong votes from people with high karma are not very anonymous. If I'm talking with someone and I strong up/down vote them (like here recently), it's pretty obvious who did it, yet not fully common knowledge, which makes the situation even more taxing on the social part of my brain than just straightforward non-anonymous voting (like on Facebook).
Yeah, that is something we've thought about, and aren't sure how to think about. We've considered straight-up requiring Strong Upvotes to be an actual endorsement that comes with a name attached. (We've also considered actually just making all votes public).
Most of the options come with some pros and cons that weigh together in subtle ways.
My subjective experience in the first 2 weeks has generally been to not know who is voting on what – theoretically I could keep track of everyone's strong-upvote strength. Maybe over time I'd come to know them well enough that I'd start to track that automatically, for now it just feels lossy enough that I don't notice.
Having a high amount of voting power basically feels like a disadvantage to me instead of a benefit, because it makes me more reluctant to exercise strong voting power. Maybe other people won't be able to tell who voted on something, but the part of my brain that worries about this kind of thing isn't really mollified by your data point.
Maybe as a compromise that ameliorates both of the problems I mentioned, consider capping the strong voting power of everyone at some low amount, like 5?
Good to know. (I do want to stay in explore-options and gather data mode rather than leap towards any particular solution but want you know I'm taking this pretty seriously).
I'm curious how you feel about solutions that are more in the direction of "make karma more illegible?" – possibly literally just randomizing +/- a few points?
There's also the option of "you can upvote UP TO your max-karma, but can choose any amount". The two reasons we didn't go with this was that it felt like asking people to making too-granular a choice every time they voted, and didn't let us move in the direction of eigenkarma. But it's still one of the more obvious things to try.
I do want to stay in explore-options and gather data mode
What kind of analysis are you thinking of doing on the data that you're gathering? I'm curious, and also pre-registration may be a good idea in situations like this to reduce bias.
I’m curious how you feel about solutions that are more in the direction of ″make karma more illegible?″ – possibly literally just randomizing +/- a few points?
Doesn't seem like it helps as much as other possibilities and the cost seems substantial (in implementation, user education, getting used to it psychologically, maybe other unforeseen consequences).
There’s also the option of “you can upvote UP TO your max-karma, but can choose any amount”. The two reasons we didn’t go with this was that it felt like asking people to making too-granular a choice every time they voted
I agree with this.
I think I may share some of the intuitions motivating this, although I like to get more data before changing course. (I also have intuitions running in the opposite direction)
I do think this is going to feel unnaturally for awhile no matter what, and want to give it time to feel more-like-normal before drawing major conclusions.
My interpretation was "if people have a rough notion of how much karma something is 'supposed' to have, then if it already starts out at, say, 9, people will be less motivated to upvote it than if it started lower".
I do this myself for a couple of reasons:
Doesn’t this attitude obviously defeat the entire purpose of the voting system?!
Can you say more about this?
My main answer is "you get more signal out of the system if people assume votes have a 'correct' amount."
Say we _don't_ have strong upvote / small upvotes. There's just a single upvote.
You see two comments: one pretty good, one great.
Say there are 100 clones of yourself using LW.
Would you rather both comments get 100 karma? Or the better comment to end up with more?
The only options here are
The 3rd and 4th option seem roughly equivalent to me.
Variable-size-votes do enable the possibility of a finer-grained result with everyone voting purely based on their individual assessments, but if everyone voted for a comment exactly based on how good they thought it was, and you _add_ those votes up, then the karma ends up being more a function of "how many people saw the comment" than "how good it is."
(Wei_Dai's comments on 'cost of voting' and 'preventing rich-getting-richer' also play into it)
If you want an idealized system wherein the expectation is "everyone votes their true appraisal of a comment's value", then the way I'd implement it would be:
1. Comment karma is hidden until you vote on it (so you don't anchor as much)
2. The karma displayed isn't the total, but something more like an average (or a function that's closer to an average than a total).
(I think this might actually be a good system, although it's a pretty big change)
My assumption here is that the outcome of options #3 and #4 are fairly similar. If Great Comment is 3x as good as Good comment, then option #3 looks something like "33% chance of upvoting" and option #4 is "upvote if it's less than about 33 karma. Possibly downvote if it's got so much karma that it's getting sorted higher than the Great Comment."
How similar the outcomes are depends on some of the precise numbers, but I think the order-of-magnitude* is about the same.
*where by order-of-magnitude I mean "base 4 order of magnitude", which is the comparison that usually seems most relevant to me.
Doublechecking what you mean by "thread-safe" (multiple people looking at the same thing at the same time making decisions before they see what other people do?)
Cool. That doesn't seem like that big an issue to me, because the system has built-in error correction – people come back later, see that it's slid past whichever direction they thought it should go, and can change their vote. It more robustly converges on Good Comment getting a proportionally correct-ish score (whereas the "roll a die in your head, upvote 33% of the time" will some non-trivial portion of the time result in things getting way-the-hell upvoted (or not) that should have been).
I should note: I don't all think this is necessarily the best approach, just, it's an approach that seems "reasonable" enough that describing it as 'defeating the point of the voting system' doesn't seem accurate.
I need to not think about this topic too much - I get wrapped around the wheel trying to find foundations and reasons for trying to be so ludicrously fine-grained for something that doesn't do very much except in cases of voting wars (which we should deal with administratively via moderators, not programmatically via game points). It makes me angry out of proportion to the (lack of) impact.
I like being able to choose no, weak, or strong votes. I don't like having to calculate (because I don't like the fact that) my strong upvote is equal to X's normal upvote. I don't like that my choices are 2 or 6, and I can't give 1 or 4 when appropriate (and if I had more karma, it would be worse).
For UI, _please_ don't overload long-click to mean something other than browser context menu. Use separate affordances, or a pop-up slider, or something.
some thoughts before i try this out:
I am worried about this thing where I both want to know: how many ppl liked a thing vs how strongly ppl liked a thing. More for posts than for comments. For posts, if I see a number like 100, I am very confused about how many ppl liked it. It seems to range between 20-50. But if... the vote power actually goes up to 15. Then... I will be confused about whether it's like... 10-50. That's... a big difference to me.
I'd almost would like it if for posts, 1 = normal 2 = strong for ppl with lower karma. And for people with more karma, it's 1 = normal 3 = strong. ? Or something that reduces the possible range for "how many ppl liked the post."
There's also a clear dynamic where people with 4-6 karma tend to check LW more frequently / early, so ... um... karma tends to go up more quickly at the beginning and then kind of tapers off, but it's like...
I dunno, it's kind of misleading to me.
Why do you top out at 16 instead of 5? I'm just ... confused by this.
Kind of wish all 'weak votes' were 1, too, and karma scores only kick in if you strong vote.
First: curious how much these concerns go away if we straight-up added a hoverover tooltip that said "this post has W upvotes, X Strong upvotes, Y downvotes, Z strong downvotes?" (this is something we've considered and don't have any obvious objections to)
Habyrka and I have chatted a bunch about some of this. I've personally leaned towards "it may be psychologically better for small-upvote to always be '1' and big-upvote is the only thing that ranges", but this was more of an intuition/guess on my part. Habryka's take was that it was pretty important to have people who've been using the site awhile and understand the culture to have a bigger small-upvote signal.
curious how much these concerns go away if we straight-up added a hoverover tooltip that said "this post has W upvotes, X Strong upvotes, Y downvotes, Z strong downvotes?"
I want to have this feature, because right now if I want this information the only way to get it is to constantly refresh my profile page to see how the karma on my posts and comments have changed, which over time lets me infer something about how many normal/strong upvotes/downvotes I got.
Personally, I like not being able to tell how many downvotes things have gotten. On the old LW, I frequently checked the percent up/down that comments and posts got, and it primed me a lot more to feel defensive or like I was in an adversarial environment. The triggered emotion is something like 'Oh, this awesome thing inexplicably got 20% downvotes; I need to be on the lookout for bad people to push/strike back against.'
Hmm. This is an interesting point, which I think I agree with (having recently been testing out this feature and getting to see some of my comments get downvoted, and having a bit of the reaction you described).
I _also_ agree with Wei_Dai's point, which is that right now we're incentivizing people to learn the system but in a delayed, confusing way. Doing something to help people learn what the system means may be good.
Unreal's suggestion of "just once, publish a list of posts and comments with their associated vote-types, so you can get a feel for what it means" is also plausible, although I note that this doesn't really make it discoverable for new users.
Hmm. Hmm. Hmm.
So possible ideas:
1. Listing _total_ votes is helpful for at least inferring roughly what a post's overall engagement is, which I think is valuable user feedback, and might get you 60%-80& of the way there. I notice that I feel way less "raised hackles" if I see my karma go down than if I see an explicit "you have been downvoted" remark.
2. Crazy idea: list total number of votes, and then list _strong upvotes_ and _strong downvotes_, based on the theory that....
It seems good to me that the people who've contributed the most to the intellectual culture and ideas on the site have 10-15 karma, relative to 5-8 like yours and mine. I'm only working on building the site, I didn't invent a new decision theory.
It also seems good to me that a newer user like you, whose written some great posts and were curated, has more influence on discussion than others who've been around a similar length of time.
Can I just have THAT number as a tooltip?
An important thing here is that we're _not_ wanting "number of people who've interact with a post" to be the dominant thing people find themselves optimizing for. That incentivizes content that is some combination of clickbaity, tribal, viral, or minimally-and-unobjectionably-good. i.e. the rest of the internet.
We want to incentivize content that thoughtful people consider and think is real good, and the central thesis here is that the natural impulse to ask "how many people liked a thing" isn't actually the best guiding star for incentivizing good content (either from a site design standpoint, or from a personal-motivation of individuals standpoint)
Also, it's kind of weird to me that I have 5 vote power given I've only really interacted with this site for... a few months? And you guys have, like, 6? 7? Are you sure your scaling is right here? :/
Well, that's part of the reason why we're increasing the scaling somewhat – if Strong Votes capped out around 5, there's even less gradation between experienced users and new/medium users.
We do want want thoughtful, good writers to have a fairly easy time getting to a voting power that feels midrange, but then room/incentivization to continue to grow. In the 1-5 strong upvote scale, that'd mean getting "3" fairly quickly, and then little gradation between people with 2000 karma and 200,000.
Random notes of my personal experience after a couple months of Strong Upvotes:
I find myself frequently wishing to have roughly 3 levels of granularity:
I want the second and third levels to be something that has to be rarer in order to be effective (mostly likely simply becoming less powerful the more you use them)
What about having users send a short private message explaining their strong vote? It's more of a speed bump than forcing someone to hold the mouse down, and it seems valuable in either case:
For a strong upvote, verbal praise is a powerful reinforcer. Consider +10 karma vs "Reading your comment helped me X, and now life is great". Nudging people to say the second without any worry of cluttering up a thread seems potentially very good.
For a strong downvote, see this post. (Could also be an argument for downvote-free zones.)
I kinda agree with Dagon that we're entering the zone of diminishing returns, but I could see this being very useful if the site was in danger of being overrun.
Trivial inconvenience alert: My regular username/password did not work on the lessestwrong site, and I had to create a new account.
Cool. We have considered doing the "strong vote requires short message" thing, and it might well be a good combination of useful + the right amount of trivial inconvenience.
Main reason not to (yet) is it's an extra layer of complexity to code, and it seems better for a few reasons to try the simpler version of the thing first.
I am worried about how easy this makes it for one individual to come along after people have stopped paying attention to the thread and massively distort the voting. Like, the combination of a strong upvote and a strong downvote could easily outweigh many people acting in good faith.
A simple thing you guys could do is freeze the scores on all the LW 1.0 content that got imported and used the old voting scheme. Otherwise the signal from those old votes may gradually get degraded by opinionated archive readers.
Initial feedback:
(will add more below as I have things to add)
I think you should give more time for the current multipliers to play out. In addition, I think having a separate button for super up-voting is better than click and holding, especially for mobile use and compatibility with different interfaces.
edit - also we ALREADY have a system for Chosen Ones to super-upvote or super-downvote things by promoting them or moderating them out of existence, I'm not sure why we need to add an extra form of super-multiplier effect.
I have some more thoughts on "what good effects precisely are we aiming for here", which I'll flesh out in the comments and/or the final blogpost when we actually deploy this change to production.
Oh no, it is the future and I did not write those comments AFAICT, or at least I'm not sure if I did.
2018_Raemon, what did you see!?
I've started removing the default strong upvote on my new comments, because it gives them more weight than I want.
We considered using another log scale, but log5 didn’t quite give us the granularity we wanted, and smaller log scales produced weird numbers that just didn’t really correspond to the effect we wanted. So we just picked some numbers that felt right.
Note that these numbers approximately follow a log2 curve.
Just for the benefit of anyone searching for information about this: one thing that's changed since this was posted is that the weak-vote strength is always either 1 or 2. Not even thousand-year-old vampires have weak/normal votes of strength > 2. (It was changed in February 2020.)
(Also: empirically I find that I almost never strong-vote on anything, and looking at the scores things are at it looks as if the same goes for LW's other high-net-karma individuals.)
3 – 25,000 karma
I think this is no longer correct, looking at the link I believe displays the code: https://github.com/ForumMagnum/ForumMagnum/blob/devel/packages/lesswrong/lib/voting/voteTypes.ts
I don't think anyone is likely to care, but it is my understanding that a new reader will see the sidecomment when they read the post and so this clears up that possible confusion.
FYI I find it almost impossible to strong vote on iPhone safari. Safari interprets double tap to mean zoom (in/out), and the hit target is very small even when zoomed in, making it too frustrating. I get that it’s not supposed to be trivial to strong upvote and I also get that phones don’t allow the same range of interactions. Not sure if there is a good solution, just bringing it up.
Update: This has now been pushed to the live site. For now, on Desktop, strong upvotes require click-and-hold. Mobile users just tap multiple times. This is most likely a temporary solution as we get some feedback about how the respective modes work under realistic conditions.
Over on our dev-site, lessestwrong.com, we have a fairly major feature-branch: Recalibrated voting power, and Strong Upvotes/Downvotes.
tl;dr – Normal votes on the test server range in power from 1 to 3 (depending on your karma). You have the option of holding down the up/downvote button for a strong vote, which ranges in power from 1 to 15.
We're looking for feedback on the UI, and some implementation details.
Flow vs Stock
This post by Jameson Quinn notes that there's two common reasons to upvote or downvote things. This is similar to my own schema:
People instinctively use upvoting to cover both Flow and Importance, and this often results in people upvoting things because they were a good thing to say in a conversation. But then later, if you want to find the most useful comments in a discussion, you end up sifting through a bunch not-actually-useful stuff.
People also often unreflectively upvote things they like, without paying much attention to whether the arguments are good, or whether it's good for the longterm health of the site. This means people who think hard about their upvotes get counted just as much as people casually clicking.
So the idea here is that by default, clicking results in a Normal Upvote. But, if you hold the button down for a couple seconds, you'll get a Strong Upvote. (And same for downvotes).
Can you technically Strong Upvote everything? Well, we can't stop you. But we're hoping a combination of mostly-good-faith + trivial inconveniences will result in people using Strong Upvotes when they feel it's actually important.
I have some more thoughts on "what good effects precisely are we aiming for here", which I'll flesh out in the comments and/or the final blogpost when we actually deploy this change to production.
Vote-Power by Karma
Quick overview of the actual numbers here (vote-power/karma)
Normal votes
Strong Votes
(We considered using another log scale, but log5 didn't quite give us the granularity we wanted, and smaller log scales produced weird numbers that just didn't really correspond to the effect we wanted. So we just picked some numbers that felt right.)
Feedback
We're still hashing out the exact UI here – in particular, the UI for helping users discover the feature. (Posts basically have little-to-know discoverability, comments have a little hover-over message).
Check out lessestwrong.com and note your feedback here. (If you created your user account recently, you may need to create an alternate account on the lessestwrong development database)