I think the percentage of "casual" users who participate on this site because they enjoy intelligent conversations on rationality-related topics while having no FAI agenda is non-negligible.
I think they're a majority, and that's the problem. There's a social norm that tolerates doing nothing about the most important problem humanity has ever faced.
It may be a net negative even if LW's mission is FAI activism as there are positive externalities of greater diversity of both discussion topics and participant opinions (less boredom, more new ideas, better critical scrutiny of ideas, less danger of community evaporative cooling, greater ability to attract new readers...)
Charging a fee to comment is not the same as banning non-FAI topics. I agree that discussion of other topics can provide instrumental value to the SIAI.
The recent implementation of a -5 karma penalty for replying to comments that are at -3 or below has clearly met with some disagreement and controversy. See http://lesswrong.com/r/discussion/lw/eb9/meta_karma_for_last_30_days/7aon . However, at the same time, it seems that Eliezer's observation that trolling and related problems have over time gotten worse here may be correct. It may be that this an inevitable consequence of growth, but it may be that it can be handled or reduced with some solution or set of solutions. I'm starting this discussion thread for people to propose possible solutions. To minimize anchoring bias and related problems, I'm not going to include my ideas in this header but in a comment below. People should think about the problem before reading proposed solutions (again to minimize anchoring issues).