Experiment: Changing minds vs. preaching to the choir
1. Problem
In the market economy production is driven by monetary incentives – higher reward for an economic activity makes more people willing to engage in it. Internet forums follow the same principle but with a different currency - instead of money the main incentive of internet commenters is the reaction of their audience. A strong reaction expressed by a large number of replies or “likes” encourages commenters to increase their output. Its absence motivates them to quit posting or change their writing style.
On neutral topics, using audience reaction as an incentive works reasonably well: attention focuses on the most interesting or entertaining comments. However, on partisan issues, such incentives become counterproductive. Political forums and newspaper comment sections demonstrate the same patterns:
- The easiest way to maximize “likes” for a given amount of effort is by posting an emotionally charged comment which appeals to audience’s biases (“preaching to the choir”).
- The easiest way to maximize the number of replies is by posting a low quality comment that goes against audience’s biases (“trolling”).
- Both effects are amplified when the website places comments with most replies or “likes” at the top of the page.
The problem is not restricted to low-brow political forums. The following graph, which shows the average number of comments as a function of an article’s karma, was generated from the Lesswrong data.

The data suggests that the easiest way to maximize the number of replies is to write posts that are disliked by most readers. For instance, articles with the karma of -1 on average generate twice as many comments (20.1±3.4) as articles with the karma of +1 (9.3±0.8).
2. Technical Solution
Enabling constructive discussion between people with different ideologies requires reversing the incentives – people need to be motivated to write posts that sound persuasive to the opposite side rather than to their own supporters.
We suggest addressing this problem that this problem by changing the voting system. In brief, instead of votes from all readers, comment ratings and position on the page should be based on votes from the opposite side only. For example, in the debate on minimum wage, for arguments against minimum wage only the upvotes of minimum wage supporters would be counted and vice versa.
The new voting system can simultaneously achieve several objectives:
· eliminate incentives for preaching to the choir
· give posters a more objective feedback on the impact of their contributions, helping them improve their writing style
· focus readers’ attention on comments most likely to change their minds instead of inciting comments that provoke an irrational defensive reaction.
3. Testing
If you are interested in measuring and improving your persuasive skills and would like to help others to do the same, you are invited to take part in the following experiment:
Step I. Submit Pro or Con arguments on any of the following topics (up to 3 arguments in total):
Should the government give all parents vouchers for private school tuition?
Should developed countries increase the number of immigrants they receive?
Should there be a government mandated minimum wage?
Step II. For each argument you have submitted, rate 15 arguments submitted by others.
Step III. Participants will be emailed the results of the experiment including:
- ratings their arguments receive from different reviewer groups (supporters, opponents and neutrals)
- the list of the most persuasive Pro & Con arguments on each topic (i.e. arguments that received the highest ratings from opposing and neutral groups)
- rating distribution in each group
Step IV (optional). If interested, sign up for the next round.
The experiment will help us test the effectiveness of the new voting system and develop the best format for its application.
Should you write longer comments? (Statistical analysis of the relationship between comment length and ratings)
A few months ago we have launched an experimental website. In brief, our goal is to create a platform where unrestricted freedom of speech would be combined with high quality of discussion. The problem can be approached from two directions. One is to help users navigate through content and quickly locate the higher quality posts. Another, which is the topic of this article, is to help users improve the quality of their own posts by providing them with meaningful feedback.
One important consideration for those who want to write better comments is how much detail to leave out. Our statistical analysis shows that for many users there is a strong connection between the ratings and the size of their comments. For example, for Yvain (Scott Alexander) and Eliezer_Yudkowsky, the average number of upvotes grows almost linearly with increasing comment length.

This trend, however, does not apply to all posters. For example, for the group of top ten contributors (in the last 30 days) to LessWrong, the average number of upvotes increases only slightly with the length of the comment (see the graph below). For quite a few people the change even goes in the opposite direction – longer comments lead to lower ratings.

Naturally, even if your longer comments are rated higher than the short ones, this does not mean that inflating comments would always produce positive results. For most users (including popular writers, such as Yvain and Eliezer), the average number of downvotes increases with increasing comment length. The data also shows that long comments that get most upvotes are generally distinct from long comments that get most downvotes. In other words, long comments are fine as long as they are interesting, but they are penalized more when they are not.

The rating patterns vary significantly from person to person. For some posters, the average number of upvotes remains flat until the comment length reaches some threshold and then starts declining with increasing comment length. For others, the optimal comment length may be somewhere in the middle. (Users who have accounts on both Lesswrong and Omnilibrium can check the optimal length for their own comments on both websites by using this link.)
Obviously length is just one among many factors that affect comment quality and for most users it does not explain more than 20% of variation in their ratings. We have a few other ideas on how to provide people with meaningful feedback on both the style and the content of their posts. But before implementing them, we would like to get your opinions first. Would such feedback be actually useful to you?
Rational Discussion of Controversial Topics
Two months ago we began testing an experimental website for Rational Discussion of Politics. Our main goal was to create a platform that would allow high quality discussion of controversial topics without resorting to any forms of censorship. The website is now ready and new members are welcome to join the discussions.
Many thanks to all the LessWrong members who have been taking part in this project.
P.S. A note to new users. A key feature of the new website is the automated recommendation system which evaluates all comments and articles based on their potential interest for each user. The recommendation system has passed the initial calibration, but its ongoing performance is sensitive to the number of user ratings per comment/article. So rating posts that you read is highly encouraged.
Measuring open-mindedness
Recently we have opened an experimental website for Rational Discussion of Politics. A special feature of the new website is an automated recommendation system which studies user preferences based on their voting records. The purpose of this feature is to enhance the quality of discussion without using any form of censorship.
The recommendation system was previously tested with the help of 30 members of a political discussion forum. The tests have shown that most user preferences can be reasonably well described by just two parameters. The system chooses the parameters (principal vectors) independently based only on the numerical data (comment ratings), but it was easy to see that one vector corresponded to the “leftwing - rightwing” and another to the “well written – poorly written” axis.
About a month ago we started discussions on the new website. This time, all our participants were LW members and the results were very different. There was relatively little variation along “well written – poorly written” axis. There was significant variation along what seemed to be the political views axis, but it could no longer be perfectly described by the conventional “leftwing - rightwing” labels. For the moment, we adopted “populares” and “optimates” terms for the two camps (the former seems somewhat correlated with “left-wing/liberal” and the latter with “right-wing/libertarian”).
The results have shown an interesting asymmetry between the camps. In the previous tests, both left and right leaning users upvoted users from their own camp much more frequently. However, one group was several times more likely to upvote their opponents than the other. Among “populares” and “optimates” the asymmetry was a lot weaker (currently 27%), but still noticeable.
In both cases our sample sizes were small and may not be representative of the LW community or the US population. Still, it would be interesting to find an explanation for this asymmetry. One possibility is that, on average, one side presents significantly better arguments. Another possibility is that the other group is more open-minded.
Can anyone suggest a test that can objectively decide which (if any) hypothesis is correct?
Rational discussion of politics
In a recent poll, many LW members expressed interest in a separate website for rational discussion of political topics. The website has been created, but we need a group of volunteers to help us test it and calibrate its recommendation system (see below).
If you would like to help (by participating in one or two discussions and giving us your feedback) please sign up here.
About individual recommendation system
All internet forums face a choice between freedom of speech and quality of debate. In absence of censorship, constructive discussions can be easily disrupted by the inflow of the mind-killed which causes the more intelligent participants to leave or descend to the same level.
Preserving quality thus usually requires at least one of the following methods:
- Appointing censors (a.k.a. moderators).
- Limiting membership.
- Declaring certain topics (e.g., politics) off limits.
On the new website, we are going to experiment with a different method. In brief, the idea is to use an automated recommendation system which sorts content, raising the best comments to the top and (optionally) hiding the worst. The sorting is done based on the individual preferences, allowing each user to avoid what he or she (rather than moderators or anyone else) defines as low quality content. In this way we should be able to enhance quality without imposing limits on free speech.
UPDATE. The discussions are scheduled to start on May 1.
Natural Selection of Government Systems
Historically, the evolution of government systems was mainly driven by violence, with invasions and revolutions being the principal agents of selection process. The rules of the game were predetermined by our environment - land was a limited resource, for which our ancestors had to compete, if only to ensure the survival of their descendants.
The 20th century introduced a game changer. As agricultural productivity in developed countries rose by orders of magnitude and natural population growth practically came to a halt, possessing a large territory stopped being a necessity. Countries with little arable land, ultra-high population density and no natural resources can now not only feed their population, but also achieve top living standards. These changes may open a fundamentally different route for societal evolution – one that would not be based on violence or compulsion.
A small thought experiment - imagine what would happen if central governments cede most powers to smaller territorial units:
- The initial result will be political divergence - with less interference from central authorities, policies of local communities will inevitably move away from the political center. The divergence will further increase as people gradually relocate to communities whose laws closely reflect their own preferences.
- Eventually, successes and failures of each community will clearly demonstrate the consequences of their policies, and the process of political divergence will be reversed. Local governments that fail to emulate the best examples will simply lose their population, who will vote against them with their feet. Just like natural selection promotes survival of the fittest, competition between different communities will promote policies that maximize human wellbeing.
- In the end, it is likely that a significant difference between communities will persist (after all, even siblings have different tastes). However, peculiar policies of each community will be a voluntary choice of its residents, and people with different ideologies will no longer have to impose their views on each other.
Unfortunately, there are serious obstacles to the successful implementation of this idea:
- It is not clear how central governments can be made to cede any of their powers. Even politicians elected on the platform of curtailing government power would be strongly tempted to renege on their promises.
- Several countries in the past were organized along somewhat similar principles – the city-states of the Ancient Greece, Northern Italy during the Renaissance period and the Dutch Republic during its Golden Age. Each achieved spectacular economic, technological and cultural development, but in all cases either the country itself or its political system eventually fell as a result of external aggression. This threat is still relevant, as many nations have not yet reached the economic or mental stage when they are no longer interested in territorial conquests.
- Most importantly, it is not clear how people from fundamentally different political camps (e.g., socialists and libertarians) can be convinced to work together for a common purpose.
Do you think these problems are solvable?
What makes you YOU? For non-deists only.
From the dawn of civilization humans believed in eternal life. The flesh may rot, but the soul will be reborn. To save the soul from the potential adverse living conditions (e.g. hell), the body, being the transient and thus the less important part, was expected to make sacrifices. To accumulate the best possible karma, pleasures of the flesh had to be given up or at least heavily curtailed.
Naturally the wisdom of this trade-off was questioned by many skeptical minds. The idea of reincarnation may have a strong appeal to imagination, but in absence of any credible evidence the Occam’s razor mercilessly cuts it into pieces. Instead of sacrificing for the sake of the future incarnations, a rationalist should live for the present. But does he really?
Consider the “incarnations” of the same person at different ages.
The Hidden Origins of Ideas
It is well known that people tend to inherit their world view toghether with their genes. Buddhists are born to the Buddhists, Muslims are born to the Muslims and Republicans are born to the Republicans. While rejecting Predestination, a XVI century catholic could be fairly certain that, unlike hell-bound pagans in the Amazonian forests, most of his descendants would also be catholics.
Naturally independent minds can occasionally break with the tradition. A catholic, finding the Pope’s stance on Predestination inconsistent with the Scriptures, might turn to Protestantism. Hence, the invention of the printing press that made Bibles widely available may have been the root cause of the Reformation. Similarly, the spread of literacy to the lower classes may have eroded the influence of the church and popularized the secular ideologies, such as Marxism.
But could it be that when we break with the traditional mode of thinking, we are driven not by superior intellects or newly acquired knowledge, but rather by something we are not even aware of? Let’s take as an example the spread of seemingly unrelated ideologies of Protestantism and Marxism.

From left to right: The european countries painted blue are those with Germanic majority, those with large numbers of protestants (>45% of all believers), and those where communists electoral vote failed to rise above 10% within the last 60 years.
While the maps are not identical, there seems to be a strong correlation between peoples’ ethnic origins, their religious histories and the openness to the communist ideas. Of course, correlation does not imply causation. However, strong correlation between our views and those of people with a similar background, may suggest that factors other than logic are responsible for them. Unless, as in my case, a similar background means smarter/ more virtuous/ more rational/ getting secret revelations from Omega/… (circle the right answer).
Rational Defense of Irrational Beliefs
“Everyone complains of his memory, but nobody of his judgment." This maxim of La Rochefoucauld rings as true today as it did back in the XVIIth century. People tend overestimate their reasoning abilities even when this overconfidence has a direct monetary cost. For instance, multiple studies have shown that investors who are more confident of their ability to beat the market receive lower returns on their investments. This overconfidence penalty applies even to the supposed experts, such as fund managers.
So what an expert rationalist should do to avoid this overconfidence trap? The seeming answer is that we should rely less on our own reasoning and more on the “wisdom of the crowds”. To a certain extent this is already achieved by the society pressure to conform, which acts as an internal policeman in our minds. Yet those of us who deem themselves not very susceptible to such pressures (overconfidence, here we go again) might need to shift their views even further.
I invite you now to experiment on how this will work in practice. Quite a few of the recent posts and comments were speaking with derision about religion and the supernatural phenomena in general. Did the authors of these comments fully consider the fact that the existence of God is firmly believed by the majority? Or that this belief is not restricted to the uneducated but shared by many famous scientists, including Newton and Einstein? Would they be willing to shift their views to accommodate the chance that their own reasoning powers are insufficient to get the right answer?
Let the stone throwing begin.
View more: Next
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d)