Perhaps we have this backwards?
If there is something intrinsically valuable about controversy (and I'm not really sure that there is, but I'm willing to accept the premise for the sake of discussion), and we're not getting the optimal level of controversy on the topics we normally discuss (again, not sure I agree, but stipulated), then perhaps what we should be doing is not looking for "more and better contrarians" who will disagree with us on the stuff we have consensus on, but rather starting to discuss more difficult topics where there is less consensus.
One problem is, of course, that some of us are already worried that LW is too weird-sounding and not sufficiently palatable to the mainstream, for example, and would probably be made uncomfortable if we explore more controversial stuff... it would feel too much like going to school in a clown suit. And moving from areas of strength to areas of weakness is always a little scary, and some of us will resist the transition simply for that reason. And many more.
Still, if you can make a case for the value of controversy, you might find enough of us convinced by that case to make that transition.
Controversial doesn't necessarily mean weird-sounding. For example, we could talk more about medicine, an area with a great deal of disagreement, without seeming like clown-suit wearing crazies. Mainstream topics should be more than enough to fill the controversy quota.
I'm worried that LW doesn't have enough good contrarians and skeptics, people who disagree with us or like to find fault in every idea they see, but do so in a way that is often right and can change our minds when they are. I fear that when contrarians/skeptics join us but aren't "good enough", we tend to drive them away instead of improving them.
For example, I know a couple of people who occasionally had interesting ideas that were contrary to the local LW consensus, but were (or appeared to be) too confident in their ideas, both good and bad. Both people ended up being repeatedly downvoted and left our community a few months after they arrived. This must have happened more often than I have noticed (partly evidenced by the large number of comments/posts now marked as written by [deleted], sometimes with whole threads written entirely by deleted accounts). I feel that this is a waste that we should try to prevent (or at least think about how we might). So here are some ideas: