LESSWRONG
LW

1 min read

Comment Permalink

I was particularly concerned about this feature and ended up pushing very hard for implementing batching and making it easy to disable before we push anything like this.

Zvi in particular has warned in the past about making karma more prominent on LessWrong, and I roughly share his worries. A bunch of initial designs of this had a display of your total karma right next to your user, which I really didn't like, because I think that would push too much in the direction of karma being a direct measure of your status, and would lead to too much social comparison and goodharting. I want new users who show up on the site to feel rewarded when they engage with content, not to feel repeatedly reminded of their relative inferiority to the senior members of the site in terms of karma. (We also decided against displaying total karma in the new user hover-preview which shows you the data a user joined, as well as the number of comments and posts they created, but very intentionally not their total karma)

I do think that giving users full control over the relevant UI elements makes a large difference here, and deals with a good chunk, though not all, of the problems with karma notifications. Batching I think also helps with a good chunk of them, by giving you a lot more granular control over how you want your behavior to be reinforced.

The things that I am most worried about going forward are:

Users ending up choosing to set too high of a frequency for their karma notifications, even though they would probably reflectively want to be notified less often
The overall culture of the site being a lot more karma driven, in a way that causes a bunch of problematic goodharding dynamics. The presence of the karma notifications might raise the mental importance of karma in a lot of users minds, who then end up judging other users by their total karma a lot more, or who end up being much more defensive in the comments that they write, because they are optimizing a lot harder for producing higher karma comments.
Causing a bunch of really good people to spend more time than they should on LessWrong, because the default setting causes them to get addicted and they never end up reflecting enough to realize that they should probably reduce their notification frequency
The initial experience of seeing karma notifications causing a lot of people to pattern-match lesswrong to a site that is out to get you, and feeling a loss of trust or sense of betrayal that makes them bounce off the site completely, before they notice the ability to turn off karma notifications (or maybe noticing it, but still thinking that some kind of trust was lost)

Showing 3 of 19 replies (Click to show all)

Vladimir_Nesov6y*20

The reversal test is with respect to the norm, not with respect to ways of handling a fixed norm. So imagine that the norm is the opposite, and see what will happen. People will invent weird things like gaging popularity based on number of downvotes, or sum of absolute values of upvotes and downvotes, when there are not enough downvotes. This will work about as well as what happens with the present norm. In that context, the option of "only upvotes" looks funny and pointless, but we can see that it actually isn't, because we can look from the point of view

... (read more)

28John_Maxwell6y

FYI, I talked to Oliver about this and he says: * The average post gets between 200 and 500 unique views in the first month, with curated ones usually getting around 2k to 5k. * Usually viewership appears to be roughly a factor 20 or 30 times the vote count.

16Kaj_Sotala6y

I think I might have posted significantly less on the original Less Wrong if it had made downvotes this salient at a time when I wasn't yet confident in the quality of my contributions, and if it didn't offer a "don't show downvotes" option.

See in context

92 Karma-Change Notifications

by jimrandomh

2nd Mar 2019

1 min read

92

Starting this weekend, LessWrong will be displaying karma notifications in the top-right corner, telling you about when you've been upvoted or downvoted. You can click the star icon to see which of your posts and comments have been voted on, and how much their score has changed.

This works a little differently from how most web sites do it. I've noticed a tendency, in myself and others, to sometimes obsessively refresh pages hoping to have gotten Likes. The rest of the LessWrong team has noticed this too. That's why on LessWrong, by default, this won't work; karma-change notifications are grouped into daily batches, so after you've checked it, you won't be notified of any additional votes until the next day.

While I hope that this prevents people from using LessWrong in ways that they don't endorse, daily batching might not be a strong enough safeguard for everyone. If you find yourself tempted to check LessWrong more often than you think is ideal, you can change the karma-notifier batches to weekly, or disable them entirely. On the other hand, if checking and posting on LessWrong is something you'd rather do more of (because you'd otherwise do something less valuable), you can set it to real-time.

As a nonprofit with no advertising, LessWrong is under a different set of incentives than most websites. We worry a lot about the trade-off between engagement and addictiveness. I want people to use the site in ways they reflectively endorse, even if that means using it less. I think LessWrong should leave that under user control, as much as feasible.

(As a reminder: Voting on LessWrong is a way of saying "I want to see more/less things like this". Upvotes are a way to tell people that their post or comment was worth their time to write, and worth your time to read. It does not necessarily indicate agreement; sometimes a good write-up of an ultimately incorrect idea is what a conversation needs. Conversely, downvotes do not necessarily indicate disagreement; sometimes a correct points is written in a way that's confusing, inflammatory or otherwise detracts from the conversation.)

As always, bug reports are welcome here, via the Intercom widget, or on the GitHub issue tracker.

Site Meta

Personal Blog

92

Mentioned in

90LessWrong FAQ

26LW Update 2019-01-03 – New All-Posts Page, Author hover-previews and new post-item

New Comment

44 comments, sorted by

top scoring

Click to highlight new comments since: Today at 7:36 PM

[-]habryka6y580

I was particularly concerned about this feature and ended up pushing very hard for implementing batching and making it easy to disable before we push anything like this.

The things that I am most worried about going forward are:

Users ending up choosing to set too high of a frequency for their karma notifications, even though they would probably reflectively want to be notified less often
The overall culture of the site being a lot more karma driven, in a way that causes a bunch of problematic goodharding dynamics. The presence of the karma notifications might raise the mental importance of karma in a lot of users minds, who then end up judging other users by their total karma a lot more, or who end up being much more defensive in the comments that they write, because they are optimizing a lot harder for producing higher karma comments.
Causing a bunch of really good people to spend more time than they should on LessWrong, because the default setting causes them to get addicted and they never end up reflecting enough to realize that they should probably reduce their notification frequency
The initial experience of seeing karma notifications causing a lot of people to pattern-match lesswrong to a site that is out to get you, and feeling a loss of trust or sense of betrayal that makes them bounce off the site completely, before they notice the ability to turn off karma notifications (or maybe noticing it, but still thinking that some kind of trust was lost)

[-]ESRogs6y300

Unless I missed it, neither this comment nor the main post explains why you ultimately decided in favor of karma notifications. You've listed a bunch of cons -- I'm curious what the pros were.

Was it just an attempt to achieve this?

I want new users who show up on the site to feel rewarded when they engage with content

[-]habryka6y490

Good point, here is why I do think the current implementation is probably worth the cost:

We've talked to a bunch of core contributors and people who are now doing great work in AI alignment and other domains, and a pretty surprising fraction of them said that they got engaged because of the karma leaderboard on the old LessWrong. Scott Garrabrant was one of the top people here, who mentioned that early on the karma system was quite important to keep him motivated and to continue working on AI Alignment.
We had the feature activated for mods only in the last three weeks, and I did notice it generally increasing my satisfaction with LessWrong by a pretty significant amount. I have it set to daily-batching, and it makes me quite happy to see when some old comments of mine are getting upvoted, and generally makes me feel more appreciated and motivated in a bunch of important ways. I've found it particularly exciting to see comments and posts of mine that are many months old still getting upvotes, which I think makes me generally better calibrated on the long-term value of writing things up and making them public, instead of just talking to people in-person which tends to have a higher immediate reward but a lower long-term reward.
I actually think it is quite valuable for users to be broadly responsive to upvotes and downvotes, and in particular want to give users more detailed analytics on when their stuff gets upvoted. Another part of the karma notification system that we are currently working on is to add a page with a time-range slider that allows you to run basic analytics on when and how your stuff got upvoted and downvoted. For many early users, being responsive to general karma I think is a good way to improve their writing and ideas, much better than click-count which I think is the standard metric that people would use if they don't have something else available.

[-]Kaj_Sotala6y260

We had the feature activated for mods only in the last three weeks, and I did notice it generally increasing my satisfaction with LessWrong by a pretty significant amount.

What's the current thinking on the way it displays downvotes? My today's karma change showed a red -7, and while I've got enough karma that I don't care that much about karma changes in general and could just shrug it off, it did feel more distinctly more unpleasant than much larger karma increases in the previous days have felt pleasant. I'm a little worried that this might make new users take downvotes even more harshly than they otherwise would.

Sites like FB tend to basically only display positive information, probably for this reason - e.g. they tell you when someone accepts your friend request or likes your comment, but they don't tell you when someone unfriends you or (AFAIK) when someone deletes your comment. I'm not sure if that's quite the right answer here - you did mention wanting to make users also more responsive to downvotes - but having a user come back to the site and run into a red negative number as one of the first things they see feels a little harsh. Maybe have the immediately-visible-number only display the sum of upvotes, but still show the downvotes when clicked on?

[-]Raemon6y101

One mild tweak could just be to not make it "red"

[-]habryka6y50

Hmm, I was worried about that, but never ended up having a negative karma day during the whole test period (probably because I just wrote a bunch of pretty straightforward meta comments).

I do find myself somewhat surprised that you ended up with a net negative balance, though statistically that is bound to happen sometimes. I can imagine that we should probably just display those in grey and not in red, since I think people naturally respond a lot stronger to negative feedback than positive feedback. But that might still not be enough. I can imagine not showing negative values and instead just showing a grey "<0" or something like that. I will experiment a bit.

[-]Raemon6y120

One important question I have is "does anyone want to see their negative karma updates?" (I personally don't have a particular desire to see it, and would want to make sure there's at least a few existence proofs of the feature being useful to someone)

[-]DanielFilan6y180

I don't like the idea that LW will tell me my daily karma change but only if it's good news.

[-]habryka6y30

I would also feel somewhat uncomfortable with this.

[-]jimrandomh6y110

It's plausibly correct to provide the option, so long as it isn't the default. (Options: Show all, show none, show only positive, show only negative. The last option being something that no one should ever use, provided only for symmetry.)

[-]Raemon6y60

I personally wouldn't bother including the last option, symmetry be damned. Choices are bad.

[-]Rob Bensinger6y100

The main thing I like about the 'only downvotes' option is that it's kind of funny and pointless. This suits my aesthetic. I could imagine trying it out for a few weeks to see what happens / to call the bluff of the part of my primate brain that thinks social disapproval from strangers is an x-risk. :)

[-]Vladimir_Nesov6y20

The main thing I like about the 'only downvotes' option is that it's kind of funny and pointless.

I feel the same about the 'only upvotes' option. Applying the reversal test, imagine that most people treat the 'only downvotes' option seriously and suggest that it should be the default, since it agrees with the usual norms of in-person conversation. Downvotes could even measure popularity if there was enough volume, in the meantime the sum of absolute values of upvotes and downvotes can play that role.

[-]Raemon6y40

I'm not sure I parse this comment.

Applying the reversal test, imagine that most people treat the 'only downvotes' option seriously and suggest that it should be the default, since it agrees with the usual norms of in-person conversation.

The usual conversation norms are "only say positive things, unless you think it's really important to give negative feedback." This maps to "people receive messages of positive reinforcement, generally don't receive messages of significant negative reinforcement unless it's really important, and therefore treat all negative reinforcement as really important."

One could argue that (in both regular conversation and also lesswrong) the role should be reversed, wherein people usually only give positive reinforcement if it's really important. But, that'd be a change from the conversational status quo, not in agreement with it.

[-]Vladimir_Nesov6y*20

When an argument goes through in the world of the opposite status quo, we can transport it to our world. In this case, we obtain the argument that "only downvotes" is not particularly funny and pointless, instead it's about as serviceable (or about as funny and pointless) as "only upvotes", and both are not very good.

[-]Rob Bensinger6y*170

If I'm having lunch with a friend, then my usual expectation is that I'll get strong compliments if they adore my clothing style, but I won't get strong criticisms if they strongly dislike it, unless I explicitly opt in to receiving the latter feedback. Most people seem to treat high-salience personal compliments as opt-out, while treating high-salience personal criticisms as opt-in. This can be outweighed if the criticism is important enough, but otherwise, criticism tends to be relatively mild and cloaked in humor or indirection.

Thinking about it in those terms, it makes sense to me to treat upvotes as similar to "person says they love my haircut" and downvotes as similar to "person says they hate my haircut." I probably want to be able to view both kinds of feedback in a time and place of my choosing, but I don't want to have the latter feedback tossed my way literally every time I open Chrome or check my email.

It might be that those norms are fine for personal style, but that we want to promote better, more pro-criticism norms in areas that matter more. We might want to push in the direction of making critical feedback opt-out, so people can (a) update faster on things that do matter a lot, and (b) perhaps get some useful exposure therapy that will make us better at receiving tips, pushback, and contrary views in the future. Mostly I'm just making this comment so folks feel comfortable talking about their preferences openly, without feeling like they're Bad Rationalists if they're not already convinced that it's useful for them personally to receive a regular stream of downvote notifications (in the world where they get a lot of downvotes).

[-]Kaj_Sotala6y160

[-]John_Maxwell6y170

I've found it particularly exciting to see comments and posts of mine that are many months old still getting upvotes, which I think makes me generally better calibrated on the long-term value of writing things up and making them public, instead of just talking to people in-person which tends to have a higher immediate reward but a lower long-term reward.

Even vote counts underestimate viewership numbers pretty drastically, don't they? I remember making comments with embedded polls where the poll got 100+ votes and the comment was sitting at +2. (And only logged-in users can vote in polls!)

[-]John_Maxwell6y280

FYI, I talked to Oliver about this and he says:

The average post gets between 200 and 500 unique views in the first month, with curated ones usually getting around 2k to 5k.
Usually viewership appears to be roughly a factor 20 or 30 times the vote count.

[-]jimrandomh6y90

We want writing posts and comments, especially posts and comments which get a positive reception, to feel rewarding, so that people will do it more often. And, to a lesser but still significant degree, we want people to use the site.

[-]Adam Zerner6y380

I'm someone who is really worried about addictiveness. I find myself doing the compulsive refreshing, and hate myself more and more every time I find myself doing it. This coupled with my just having finished reading Digital Minimalism made me feel really worried as I first started reading this post. But once I reached the point where I realized the team was aware of the problem of addictiveness and gives users a way around it... I just felt a strong feeling of warmth towards LessWrong. hearts

[-]Yoav Ravid6y200

karma-change notifications are grouped into daily batches, so after you've checked it, you won't be notified of any additional votes until the next day

brilliant.

I was worried in the first paragraph (for the same reasons you brought up), but was relaxed by the second :)

[-]the gears to ascension6y170

Hey cool. this is the sort of reward I need to enjoy a site enough to use it.

[-]ozziegooen6y120

This feature seems pretty useful, and I really appreciate how you put thought into not making it too addicting. Having good incentives seems like a good way of allowing our community to "win", and I'm happy to see that pay off in practice.

[-]SarahNibs6y110

These notifications just showed me a very old thread I did not know existed. The thread made me laugh. Thank you, I think. https://www.lesswrong.com/posts/yffPyiu7hRLyc7r23/#okLFFnQhgTzqANvQy

[-]John_Maxwell6y*110

Cool feature! I've also noticed myself doing this.

A trick I've found for making behavior changes like this: Start with a "dopamine neutral" change that sets up a behavioral pathway for later changes. In this case, the "dopamine neutral" change is making notifications real-time. After a while, you unlearn the behavior of looking at your user page, because you're getting that info through notifications. Then you can slow the notifications down. The risk of setting them to daily or weekly right away is that you never unlearn the behavior of going straight to your userpage to get the latest changes.

Probably overkill for this use case, but the general pattern can be useful in other contexts. Example: Resolve to only open your web browser through a command line script. This is close to dopamine neutral. But once you've got that behavioral hook embedded, you can modify the script so that it forces you to wait 10 minutes, or asks you some questions about your intentions, or gets you to specify a whitelist of domains you will visit, or whatever. Then opening your browser through some other method serves as a Schelling fence you know not to cross. (If you find yourself making "just this once" modifications to the script, you could design some policies for when modifications can be made.) I've gotten a lot of mileage out of building systems like this.

[-]Matt Goldenberg6y40

Can you explain this strategy more? In theory a random reinforcement schedule shouldn't create any less dopamine when rewarded, and should be the most resistant to extinction. I'm having trouble understanding what you mean by dopamine neutral I think.

[-]TheWakalix6y170

I think this is the idea: people can form habits, and habits have friction - you'll keep doing them even if they're painful (they oppose momentary preferences, as opposed to reflective preferences). But you probably won't adopt a new habit if it's painful. Therefore, to successfully build a habit that changes your actions from momentary to reflective, you should first adopt a habit, then make it painful - don't combine the two steps.

[-]John_Maxwell6y70

Checking your userpage and checking your karma notifications are both random reinforcers, ergo switching from one to the other is dopamine neutral. Step one is to extinguish the behavior of checking your userpage by making that dopamine neutral behavior swap. Step two is decrease notification frequency.

[-]Vladimir_Nesov6y90

I'd appreciate the feature that restricts the notifications to the votes on comments posted at most X months ago (with X configurable in the settings). As it is, I'll mostly get noise, notifications for the comments posted 8-11 years ago that I'm not currently learning from. (At least I expect this to be the case and the first batch of notifications supports this.)

[-]Raemon6y40

Seems plausible. I do think it'd make sense to check in if this still seems useful in a week or so after getting a sense of the typical use case.

(I've personally found it fairly valuable to notice how much traction older posts get, which gives me a better idea of how much value they generate across time)

[-]Vladimir_Nesov6y40

No ancient updates for the previous week, several for this week. An alternative to removing old notifications is to prepend entries in the list with recency, like "13d" or "8y", and sort by it.

[-]Raemon6y20

A few months later (mostly just focusing on other priorities), curious how this now feels to you.

[-]Vladimir_Nesov6y*20

I think updates were less frequent recently (e.g. zero updates from last week). This should still happen for some people, though there is maybe only about a hundred users with similar number of ancient comments.

[-]ChristianKl6y80

Given that the batches are daily I would like to see the time batches the way they are on StackOverflow, individual batches can be expanded/contracted.

When hovering over a comment it would be great to see the text of the comment as a popup.

[-]Jameson Quinn6y60

Minor point: on the settings page, the order of options for updates is currently "disabled, daily, weekly, realtime", when "disabled, weekly, daily, realtime" would make more sense.

[-]Davis_Kingsley6y60

Great feature, I really like this.

[-]Pattern6y40

(As a reminder: Voting on LessWrong is a way of saying

Aside from this being mentioned in Meta posts, is there a general guide to things like that, or are Meta posts that guide? (Despite not being/containing a Sequence.)

[-]Gordon Seidoh Worley6y30

I appreciate what you're doing here trying to protect us, but I'd also really like a way to get the data more frequently. I understand there's probably a lot of reasons you want to make this hard, but if it's easy to tweak per user it'd be nice if I could do something like send a support request to get my frequency cranked up to once every 5 minutes or something reasonably real-time that doesn't put a bunch of strain on the system.

Basically, I know I can trust myself with this and would like it, understand why you would want to make it very hard for almost everyone to get access to it, and so just want to put out a feeler to see if super-hidden options are a possibility, even if it means I have to add the code myself and get you to flip the config for my user in the database.

[-]jimrandomh6y90

There is a real-time setting, which shows you everything since the last time you looked. It just isn't the default.

[-]Ben Pace6y50

Yeah, if you open up the new karma button, you’ll see a ‘Change Settings’ button.

[-]Gordon Seidoh Worley6y50

Oh, thanks! From the post and the comments I thought such a feature didn't exist on purpose!

[-]Dagon6y20

Would it be possible (and reasonable) to display the "external karma" on posts as well? Seeing the number excluding my own default upvote carries more information than the current display.

[-]habryka6y50

I think that would be pretty confusing, since posts and comments are indeed sorted by their total karma, not their external karma. And displaying a duplicate number that is almost the same seems more likely to confuse people. I would be pretty hesitant to do this, but it would obviously not be impossible if enough people want it.

Moderation Log