Comment Permalink

That would make it very very easy for the halo effect to kick in.

I interpret you as meaning that if some had 'flash' their posts would be considered as better than their actual merit (correct me if I'm misinterpreting). If that were to be the case would there not be an equal or stronger effect from a poster having a good reputation or a gazzilion karma (like Luke, Alicorn or Eleizer) under the current system?

There would also be the question of whether if this effect exists it is sufficient detrimental to outweigh the motivational benefits.

See in context

27 How to incentivize people doing useful stuff on Less Wrong

by lukeprog

27th Sep 2011

3 min read

27

Currently, LWers get +1 karma for a comment upvote, and +10 karma for a main post upvote. But clearly, there are other valuable things LWers could do for the community besides writing comments and posts. Writing isn't everyone's forte. Why not award karma for doing productive non-writing things? It's probably not optimal that karma and the community status that comes with it are awarded only for the thing that myself and a few other people are good at. For example, I really wish LW could award karma to programmers for improving LW.

The challenge is doing it fairly, in a way that doesn't alienate too many people. But there might be a workable way to do this, so let's explore.

Perhaps tasks could be assigned karma award amounts by LW editors (Nesov, Eliezer, Louie, etc.), or even just one person who is appointed as the Karma Genie.

Examples:

Write a 5-page document describing how to use the Less Wrong virtual machine to hack new features into Less Wrong. 900 points.
Add a Facebook 'Like' button to the left of the up-down vote buttons on every post. 700 points.
Collect PDFs for every paper on debiasing thinking error X, upload the ZIP file to mediafire. 700 points.
Write a single-page introduction to The Sequences that makes them easier to navigate and see the value of. 800 points.
Launch a new LessWrong meetup group and hold at least three meetings. 1200 points.

Another possibility is to do what what.cd does with upload ratio. The more you upload, the better your upload ratio. But, you can trade in megabytes of your 'uploaded data' count for 'requests'. For example, if you wish somebody would upload RareAlbumX, you can trade in 500 megabytes of your uploaded data count. This lowers your upload ratio, which can be rebuilt by uploading more, and incentivizes someone to upload RareAlbumX. If somebody uploads RareAlbumX, the 500 megabytes (minus a small tax) is added to their 'uploaded data' count, which raises their status and privileges in the community.

A what.cd-inspired system for LW would look like this: People could write a paragraph or two outlining the project they'd like to see done. Then, people could 'donate' karma to the project's karma award amount. If I really wanted to see somebody add the Facebook 'Like' button to LW, I could donate 2000 of my karma points to the award, and other people could donate some of their karma to that project, too. Maybe the award would eventually rise to 5000 karma points - a pretty damn good reason for some programmer on LW to write the code and make it work.

In this scenario, editors would not dictate how valuable certain projects are, but would instead merely verify that the completed work met the specifications of the project. Editors would also have 'veto' power on projects. So, if somebody proposed a project to add a dancing Barney GIF to the front page, but LW editors decided they didn't want this, they would kill the project and any karma already donated to it would be returned to the original donors. Or better yet, projects would have to be approved before they showed up on the site.

One difficulty here is that karma ranking boards still wouldn't optimally reflect status. Suppose PersonA did a ton of good work on Less Wrong and got 10,000 karma, and then donated all of that karma toward making some great project highly incentivized. This person has done two wonderful things - all the wonderful work and incentivizing more wonderful work - but they have 0 karma to show for it. On the other hand, it may be fair enough that people who accumulate karma can have more power over which particular good things are likely to happen as a result of karma incentivization.

Also, there should probably be more incentives for gaining karma in general, as there are incentives for gaining upload ratio on what.cd. Rising through the karma ranks should give users special privileges that are actually desired.

Basically, there are tons of useful projects that people could be doing, and LessWrong cannot possibly be currently optimized for incentivizing those projects. So... what's the best solution we can come up with?

Site Meta

Personal Blog

27

New Comment

60 comments, sorted by

top scoring

Click to highlight new comments since: Today at 11:44 AM

[-]fubarobfusco14y360

Am I the only one who thinks that linking this site to Facebook (or any other similar site) is a really bad idea? It sounds like a recipe for an Eternal September.

[-]vi21maobk9vp14y140

Also, it gives Facebook full access to personalized LessWrong-browsing patterns.

[-]dbaupp14y70

To add to that, there are also significant privacy concerns (this too) with loading anything from the Facebook servers (including the "Like" buttons).

[-]Tripitaka14y30

Well, in Germany one site (www.heise.de) made some nifty code to circumvent the specific problems of the like-buttons; it is open source and here available. It uses a two-click-solution; first you click on a button similar to the FB-ike-button; then you are forwarded to a second page with the real button.

[-]vi21maobk9vp14y20

Facebook tried to argue that doing this violates its ToS (not doing this would violate law in many countries).

http://cyberghostvpn.com/blog/2011/09/newsflash-facebook-doesnât-like-privacy-2/

Using a distinct button for first click seems to be OK.

[-]dbaupp14y00

Ah, that's pretty neat.

Although, I guess that'd be a trivial inconvenience, and so there would be far less point in having the buttons in the first place.

[-]FiftyTwo14y30

I respectfully disagree. Because we have no major advertising campaign or other entry point most sharing of articles and getting people interested in the community is 'peer-to-peer' i.e. someone shares an interesting article with their friends or sends a friend a specific article they may be interested in. Facebook is an ideal medium for doing this as people of all social groups use it and it is a recognised format.

I personally have introduced many friends to LW via this method, and it is the primary way that people of college age communicate and pass on information. I don't think I'm particularly unusual in that my friends share, re-post and discuss articles on political and scientific topics with great frequency.

Even if we don't use the 'like' button, there are some changes that would make it easier to share. Aesthetically, currently if I post an article link I get an ugly text preview of less wrong tagline, rather than a summary/beginning of the article or a logo. This leaves us with a credibility gap when compared to other 'mainstream' news sites. [If someone can make the necessary changes to fix this issue I offer 100 of my karma]

[-]Teal_Thanatos14y30

Hi, I believe this site is for the improvement of all mankind.

Facebook may currently be a less than intelligent social networking website. But are we not here to Raise the Sanity Waterline? I can't remember which article of Lesswrong indicated that it may be prudent not to enhance the upper reaches of Rationality, but to improve the lower end so that there are more people available to enhance the upper end. ( I believe there was an article (not This One but another) that went into detail on this.

While there may be downsides to facebook viewing, I would like to point out it can be negated reasonably. (Or unreasonably in my example:) Just by having a cookie on that facebook link so that if a non-member clicks it, it transparently prevents them from signing up for a week. As many people from facebook will immediately like the article before posting a reply, many of the 'quick responders' will be eliminated... unless they come back in a weeks time.

Alternatively, just manage the facebook like to only go to an article without comments, this prevents the 'easy' facebookers from signing up and continuing, while those with a little more wit can parse back the URL to go to the main site.

Lesswrong has the brainpower to come up with some ideas to manage this, and I think it'd make a great community project to sort out how to do so.

[-]lessdazed14y20

Channels to particular articles would be a good idea.

[-]lukeprog14y00

Too bad there's not a Facebook that is only open to scientists and mathematicians!

[-]lessdazed14y30

To what extent is there a network of frequent wikipedia editors?

[-]fubarobfusco14y30

Can we send them to "Knowing about biases can hurt people" first?

[-]lessdazed14y40

I think the question is more which posts are good introductions and screens. So even if that is the best one to share on wikipediabook+ or whatever it is, others would also be worthy. The question is which ones are good enough, not which one is best.

Different posts would be appropriate for different places, simple ones linked to on facebook might be too accessible - I'm thinking of "Are Your Enemies Innately Evil." The last thing we need is for everyone on facebook to read that post one September 11th. "Mundane Magic" might be too complicated for many people on facebook, and serve as a good screen - I'm not really sure.

I agree that your choice is good and also suggest the following:

http://lesswrong.com/lw/uy/dark_side_epistemology/ http://lesswrong.com/lw/2as/diseased_thinking_dissolving_questions_about/ http://lesswrong.com/lw/j3/science_as_curiositystopper/ http://lesswrong.com/lw/h1/the_scales_of_justice_the_notebook_of_rationality/ http://lesswrong.com/lw/hu/the_third_alternative/ http://lesswrong.com/lw/gz/policy_debates_should_not_appear_onesided/

[-][anonymous]14y10

Google+ seems to still have better demographics than Facebook.

[-]FiftyTwo14y00

That would be an anti-social network.

[-]Shmi14y170

Luke, it is rather unfortunate that you offered a problem (How to incentivize people) and a single potential solution (use karma) in the same post, committing what seems to be a rather basic violation of rationality, something one would hardly expect from an expert in the matter.

Now, as you can see from the comments, nearly everyone has concentrated on discussing your solution, as opposed to offering their own, and the utility of your post has been greatly diminished as a result.

Then again, maybe it was your intention all along, in which case I would appreciate your thinking on the matter. (For example, maybe there was some other post where plenty of options were discussed and karma use was agreed to be the way to proceed.)

[-]PhilGoetz14y120

For example, I really wish LW could award karma to programmers for improving LW.

It can't? The programmers of LW should be incentivized to implement awarding karma to programmers. (Although I would deduct karma from whoever replaced the "Reply Permalink Edit" etc. with icons.)

Suppose PersonA did a ton of good work on Less Wrong and got 10,000 karma, and then donated all of that karma toward making some great project highly incentivized. This person has done two wonderful things - all the wonderful work and incentivizing more wonderful work - but they have 0 karma to show for it.

They have real-world karma to show for it. But if that isn't enough for you, they could instead invest their karma in the project, and get back some fraction of the karma voted to the project.

[-]Raemon14y110

Typically, if someone does something useful for less wrong, they make a top level post about it, and they get karma for that post.

[-]Jack14y90

One difficulty here is that karma ranking boards still wouldn't optimally reflect status. Suppose PersonA did a ton of good work on Less Wrong and got 10,000 karma, and then donated all of that karma toward making some great project highly incentivized. This person has done two wonderful things - all the wonderful work and incentivizing more wonderful work - but they have 0 karma to show for it.

This is the difference between us and what.cd. There are so many albums in the world filling a request might mean satisfying the desire of only one or two people. But major contributions to site code are collective goods. Because they are collective goods people will often be unwilling to sacrifice karma and will instead wait for someone else to fund the projects. Unless we're going to name site features after the posters who fund them ("The Lukeprog Lawrence, KS Meetup Group!") it would be more effective to have a public funding model.

Posters should have a set monthly "Karma Vouchers" that they can spend on requests. When the request is fulfilled the person who fulfilled it is accorded in real karma whatever the task was given in vouchers. The amount of vouchers one gets can be a karma incentive as well- say you have to get to 500 karma before you get any and after that it increases occasionally as you level up. You cannot claim your own vouchers. Karma vouchers can also be put against tasks (i.e. you can downvote with the vouchers) for an easy way of disqualifying unpopular site features and preventing people from gaming the system.

[-]lessdazed14y60

Unless we're going to name site features after the posters who fund them

Chas v'shalom! I didn't leave Judaism to be part of a community in which everything is named after a donor! Do you have any idea what that's like?!

[-]NancyLebovitz14y10

Maybe you'd better tell us what it's like. I was never involved in Judaism enough to see that downside.

[-]lessdazed14y00

http://www.youtube.com/watch?v=o-Zm54fIzbk

[-]vi21maobk9vp14y10

Is there any incentive to hoard karma? Maybe we need to keep two values: Earned Karma and Karma Balance. In that case the only cost of sacrificing karma is opportunity cost, but I guess there will be only so many projects.

A more interecting question about up/down-votes is whether a person can change one's mind after using karma/voucher in voting. On the one hand, LW should allow people to change their mind; on the other hand, it may create some strange dynamics if someone wants to support more projects than they can with the current amount of karma.

[-]Oscar_Cunningham14y80

Suppose PersonA did a ton of good work on Less Wrong and got 10,000 karma, and then donated all of that karma toward making some great project highly incentivized. This person has done two wonderful things - all the wonderful work and incentivizing more wonderful work - but they have 0 karma to show for it.

We can fix this by having karma be multiplied when it is donated. So if I donate 100 karma to see some task completed that will reward the person who does that task with 200 karma. Another idea (that I prefer) is that the karma bounty you post is returned to you (as well as given to the other person) when a task is completed.

Also, there should probably be more incentives for gaining karma in general, as there are incentives for gaining upload ratio on what.cd. Rising through the karma ranks should give users special privileges that are actually desired.

On stackexchange style sites users are awarded with the power to do site admin tasks as a reward for tasking the karma ladder. For instance 2000 karma nets you the ability to edit others post (helpfully correcting broken links). Obviously there are still consequences if you use your powers for evil.

Independent of modifications to the karma system, more users should be given such editting powers. In particular the sequences could do with being more interlinked, and I would happily do this given the chance. Putting links to the next post in the sequence at the end of every post would increase their addictiveness substantially, pulling in more readers.

Finally, it occurs that making such modifications might use more work than simply making the changes that would otherwise have rewards posted for them.

[-]Vaniver14y50

Independent of modifications to the karma system, more users should be given such editting powers.

Emphasis mine. I hope this was intentional, because man did that make me want editing powers.

[-]jsalvatier14y30

Yes! Stackexchange/overflow is really well designed. There are many things which LW (and for that matter other most community sites) could learn from SE. Editing posts is a good one. They also do a really good job of making tags work well. We have a couple of editors but we could really use more.

[-]lukeprog13y00

What other lessons could LW learn from Stackexchange/overflow?

[-]jsalvatier13y10

Here are some things that I suspect make tagging on SE work well:

Searching by tags works well
You can mark some tags as favorites so that they are prioritized or excluded for you
Tags prominently displayed
Tag suggestions based on commonly used tags/words used in your post : seems like a good idea
Lots of people can retag posts : seems like a good idea
Tag wiki pages: good idea, probably would not take much work
Information about 'related tags'
Could just be that there are so many disparate topics that tags are more important

[-]Teal_Thanatos14y00

I'd like to see that. throw in an editor that allows you to suggest changes, then two others have to approve it for it to be actioned. Follow this up with the post in question having a link at the bottom 'Possible Changes Pending' so that others can be notified and then power it up with a Karma connection

(Formula:(PostKarma/10/Editors changes) rounded up to one would give a single editor 5 points for a fifty point article, five editors with approximately equal changes one karma each... or 1 each for 4 and two for the fifth editor who contributed between 20% and 40% of the approved editor changes)

[-]Jack14y50

Do we think the 10:1 ration between top-level main posts and discussion posts/comments accurately represents the differing value of those contributions? The question matters more the more value we give to karma.

[-]shokwave14y60

Maybe it doesn't accurately represent the differing value (for instance, I think of discussion as anywhere from 1/5th to 1/3rd of main), but the downvotes being worth 10 as well is critical - it keeps less-approved of things well off the front page, which is important for newcomers.

[-]MarkusRamikin14y40

Why not just have programmers post what they've done and let people upvote them for it?

Somehow I'm suspicious of any possibility of anyone's karma changing for other reasons than upvotes/downvotes.

[-]Caspian14y00

That is a good base to compare the karma donation proposal to.

Some differences in the proposal:

The task posts are not specified to be in the main post area.
The votes cost the voter karma - in terms of karma, they are zero sum.
The votes can be worth more than 10 karma.
The voter has to pick the vote value rather than use a fixed value.
The vote value is limited by the voter's total karma.
The votes are pre-committed to, after the task description is made but before the task is completed, and don't apply until it's completed.
Down votes are not available.
The example task descriptions are much shorter than typical posts.
The system needs coding rather than using the existing code.

Not sure if I've missed anything major, some of those differences only occurred to me after I thought I'd finished (e.g. lack of down votes)

[-]khafra14y00

Changes to the user interface already get posted. Most changes to parts other than the user interface would be unutterably boring to many people, and do not contribute very specifically toward the goal of LW.

[-]Vaniver14y40

Karma markets are a good plan. I approve.

[-]shokwave14y30

Before we have any more discussion, I highly recommend committing yourselves to using "karma charity", "donate karma", "receive karma" instead of "karma market", "pay karma", "paid with karma". It is aesthetically pleasing - juxtaposing karma with capitalism just grates on my sense of taste. It also has obvious framing benefits which we should not be above using - if the idea of participating in karma charity is attractive to one good programmer where buying and selling in a karma market is not, Less Wrong garners that many more improvements.

One difficulty here is that karma ranking boards still wouldn't optimally reflect status. Suppose PersonA did a ton of good work on Less Wrong and got 10,000 karma, and then donated all of that karma toward making some great project highly incentivized. This person has done two wonderful things - all the wonderful work and incentivizing more wonderful work - but they have 0 karma to show for it.

Solution: anonymise karma donations, announce the total pool of 10,000 karma, then on project completion return the largest single donation to its donor as well as giving it to the recipient. In PersonA's case, they lose no karma. This is deserved, because it is a generous move on PersonA's part. In usual usage, it will contribute slightly to karma inflation and encourage larger than usual prizes (think "closest guess to 2/3s of the average of all guesses wins" but growing instead of shrinking) without allowing people to game the system.

What's to stop Eliezer from donating 150,000 karma for anything he wants done, comfortable in the knowledge that he will receive his full karma donation back? Nothing except that this will drastically decrease his future power by massively devaluing karma. Anyone with enough karma to pull off that manipulation has too much invested in their karma to squander it.

[-]lessdazed14y30

Maybe the award would eventually rise to 5000 karma points - a pretty damn good reason for some programmer on LW to write the code and make it work.

“A soldier will fight long and hard for a bit of colored ribbon.” --Napoleon

I think more effort should be given to parceling out sub-assignments. The following is too big a project:

If somebody wants to summarize the literature on choice modelling or AI preference elicitation/learning from the point of view of what's most likely to be useful for CEV, that would be awesome. But I am very, very skeptical that anybody will actually do that.

[-]TrE14y30

Why not introduce some sort of colored digital ribbons along with Karma points for completing "quests"?

[-]lukeprog13y10

I like this. Like the 'percent complete' markers on OkCupid or Mint.com or a bunch of other sites. Percent-complete could dictate one's user class.

[-]FiftyTwo14y00

Some sub-reddits do 'flash' which is phrases or small images next to peoples user names. (E.g. r/fitness puts peoples areas of interest, gameofthrones puts small house crests).

Perhaps we could do something similar as a reward for major contributions, say a gold star, or even a paper clip?

[-]dbaupp14y00

That would make it very very easy for the halo effect to kick in.

[-]FiftyTwo14y00

There would also be the question of whether if this effect exists it is sufficient detrimental to outweigh the motivational benefits.

[-]dbaupp14y00

If that were to be the case would there not be an equal or stronger effect from a poster having a good reputation or a gazzilion karma (like Luke, Alicorn or Eleizer) under the current system?

Maybe (I think this debate has occurred several times before), but the total karma of a user is not visible on every post, and likewise with their reputation.

[-]lessdazed14y00

DING!

[-]TrE14y00

What do you want to tell me with that? Other than factual information, I can't draw much from it. Do you want to say that

a) this sort of conditioning is bad and should stay away from lw

b) it works in games and thusly seems like a good idea

c) nothing further

[-]PhilGoetz14y30

Are there people willing to purchase LessWrong karma with Second Life dollars, World of Warcraft money, or even real US dollars?

If it were possible to sell LessWrong karma for a dollar a point, some people could quit their jobs and write for LessWrong full time.

One problem is that this would in itself probably devalue karma against the dollar.

An alternate system would be to add another button next to 'thumbs up' that would use paypal to pay people a dollar directly for a clever comment or $5 for a post.

(This raises an interesting question: Would the karma system be more informative if we had no downvotes? As it stands, a karma of +/- 2 is usually a strong signal; but for a controversial comment, that doesn't rise above the noise.)

[-]Raemon14y210

If Karma were worth any money worth worrying about, then the fact that Karma is an easily gamed system would quickly become a problem.

[-]tenshiko14y50

LW Karma is in all objectivity worth less than a dollar a point. You can earn twenty karma just for a statement being particularly witty even if it's fairly obvious to the clever in the context of the post made, such as saying "I'm not!" in response to a post characterizing LWers as contrarian. Then we're asking where the money comes from -- singinst can't have that kind of money in its coffers, they're not just going to ship $100 to anyone who's lurked on the site long enough to rack up a hundred points. At this value I'd peg karma closer to a cent.

[-]vi21maobk9vp14y10

The question is the reverse - whether there exist people who would like to get LW karma as an acknowledgment of expected positive effect of their donation to SingInst.

[-]Teal_Thanatos14y30

if the downvotes were to be changed, I'd like to see more along the lines of Upvotes/Downvotes/Total Karma. Controversial posts would still have reasonable numbers of up/downs and would be visible thus, but the total karma would reflect the overall feel of the post. This information is all recorded in the db anyway.. Though to be honest, as a standard user I would hardly be interested in this information at this point. so please, make it optional.

[-]vi21maobk9vp14y20

Showing more parameters will lead to different perception of post karma, and so it should be separately discussed.

Currently, many people think along the lines "I would upvote it from 2 to 3, but not from 4 to 5, and from 10 I will downvote it to 9". So, even a non-controversial post can accumulate some upvotes and downvotes if some people find it overrated and some find it underrated.

[-]Jack14y30

Perhaps tasks could be assigned karma award amounts by LW editors (Nesov, Eliezer, Louie, etc.), or even just one person who is appointed as the Karma Genie.

Some kind of democratic or market mechanism would more accurately reflect the value of tasks.

Also, there should probably be more incentives for gaining karma in general, as there are incentives for gaining upload ratio on what.cd. Rising through the karma ranks should give users special privileges that are actually desired.

The top contributors list on the sidebar could link to a page with the full rankings.

[-]DuncanS14y20

I think the Karma value of a post is quite useful. The Karma value of a person is less so - it's a combination of how long they've been around, and how good their posts are considered to be. As long as you have enough Karma to post a main article if you want, you have enough Karma, imo. Having said that, I do take more notice of posts from people who I recognise, or have something of a track record. We'd lose that information if Karma became something else.

There is of course the well-known incentive of changing the relative probabilities of a good or bad singularity - you would think that ought to be enough for most people.... Actually it isn't - we tend to do things like less wrong in order to mix with a community of like-minded people. I think encouraging that aspect stands most chance of success.

[-]tenshiko14y20

The thing is that in the current karma model, karma simply can't be treated as currency, because every time someone upvotes something, that karma is drawn anew from the aether, it's not a transfer of existing electronic karma bills or anything like that. You can't even say there's an infinite bank it comes from, because there's not. Karma currently exists only as a ranking, and although that's a goal in itself for some people, I'd say it's not for everyone.

[-]handoflixue14y00

That's true of upload ratio as well, but that doesn't seem to cause issues for what.cd

[-][anonymous]14y10

Why not just have people who are good at writing write up a top level post explaining why non-writing contribution is important, and why person X should be thanked for doing it. They then give the article to the person in question to post (the article is still signed by the author)? The article is hopefully up voted (10 karma each up vote adds up quick), either because people like how its written or agree that it indeed is a very important contribution for other reasons.

Also a feature that I'd be willing to donate karma for is co-authorship, where the karma gained by a top level post can be split between the authors. Bonus if they can do so in a way in which they can customize how much of the karma share or burden of the article each of them will get.

This is worth about half my current karma to me.

[-]Teal_Thanatos14y10

I don't think that's a really good idea. It doesn't exactly fill me with desire to have more articles which are not directly related to rationality. Besides, having a whole bunch of articles about explaining non-writing contribution is important and person X should be thanked is going to get old very quickly.

Co-Authorship however is pretty good! it'd help people who get beta'd before posting.

[-]jsalvatier14y10

Glad others are interested in this!

[-]Alexandros14y00

Implementing this is relatively straightforward, if not easy. Make a subreddit, where people request improvements. Karma from the upvotes on those requests goes to whoever successfully implements said improvement. No reason to make karma less legitimate by having admins hand out arbitrary chunks of it.

[-]Clarica14y00

If you want to increase participation on Less Wrong, converting irrational thinkers to rational thinkers is the only way.

Less wrong does not have a mandate to educate the merely interested, probably because most of that kind of interest seems temporary. The educational efforts that seem required are rarely intrinsically rewarding. And the bar for valued participation is so high, it discourages the unprepared.

Less Wrong seems to be trying to create plenty of tools for self-education on the subject of rationality, so I don't know that a mandate to educate individuals is necessary... But I do think that anyone who is compelled to reply someone at a different level of education on a subject than themselves ought to feel compelled to point out where the energy for contribution/education should actually go, instead of merely curbing the enthusiasm to participate.

Moderation Log