Are you suggesting folks can't be trusted to reliably identify genuinely high-quality opinions that disagree with theirs?
What can we learn from this thread?
http://lesswrong.com/lw/2sl/the_irrationality_game/
You're advocating this as a good thing?
The OP talks about folks who "like to find fault in every idea they see". Assuming this is valuable, there are two ways to have this kind of person: be this kind of person naturally, or unnaturally in order to win an award.
Keep in mind that the award's specifications can be changed, for example, "best civil disagreement with LW majority" or "changed the most minds among LW users".
(Anybody is welcome to copy/paste/edit that post and run it again, probably in Main because the less casual nature of Main discourages accidental failure to read the rules. Also, I noticed that a lot of the rules weren't really necessary because people did reliably play in the spirit of the game; most of the rules are along the lines of 'don't cheat'. So if you re-run it you might want to remove a lot of the text. FWIW I'd upvote it and probably make a lot of comments.)
I'm worried that LW doesn't have enough good contrarians and skeptics, people who disagree with us or like to find fault in every idea they see, but do so in a way that is often right and can change our minds when they are. I fear that when contrarians/skeptics join us but aren't "good enough", we tend to drive them away instead of improving them.
For example, I know a couple of people who occasionally had interesting ideas that were contrary to the local LW consensus, but were (or appeared to be) too confident in their ideas, both good and bad. Both people ended up being repeatedly downvoted and left our community a few months after they arrived. This must have happened more often than I have noticed (partly evidenced by the large number of comments/posts now marked as written by [deleted], sometimes with whole threads written entirely by deleted accounts). I feel that this is a waste that we should try to prevent (or at least think about how we might). So here are some ideas: