I'm worried that LW doesn't have enough good contrarians and skeptics, people who disagree with us or like to find fault in every idea they see, but do so in a way that is often right and can change our minds when they are. I fear that when contrarians/skeptics join us but aren't "good enough", we tend to drive them away instead of improving them.
For example, I know a couple of people who occasionally had interesting ideas that were contrary to the local LW consensus, but were (or appeared to be) too confident in their ideas, both good and bad. Both people ended up being repeatedly downvoted and left our community a few months after they arrived. This must have happened more often than I have noticed (partly evidenced by the large number of comments/posts now marked as written by [deleted], sometimes with whole threads written entirely by deleted accounts). I feel that this is a waste that we should try to prevent (or at least think about how we might). So here are some ideas:
- Try to "fix" them by telling them that they are overconfident and give them hints about how to get LW to take their ideas seriously. Unfortunately, from their perspective such advice must appear to come from someone who is themselves overconfident and wrong, so they're not likely to be very inclined to accept the advice.
- Create a separate section with different social norms, where people are not expected to maintain the "proper" level of confidence and niceness (on pain of being downvoted), and direct overconfident newcomers to it. Perhaps through no-holds-barred debate we can convince them that we're not as crazy and wrong as they thought, and then give them the above-mentioned advice and move them to the main sections.
- Give newcomers some sort of honeymoon period (marked by color-coding of their usernames or something like that), where we ignore their overconfidence and associated social transgressions (or just be extra nice and tolerant towards them), and take their ideas on their own merits. Maybe if they see us take their ideas seriously, that will cause them to reciprocate and take us more seriously when we point out that they may be wrong or overconfident.
OTOH, I don’t think group think is a big problem. Criticism by folks like Will Newsome, Vladimir Slepnev and especially Wei Dai is often upvoted. (I upvote almost every comment of Dai or Newsome if I don’t forget it. Dai makes always very good points and Newsome is often wrong but also hilariously funny or just brilliant and right.) Of course, folks like this Dymytry guy are often downvoted, but IMO with good reason.
I think we can see now how the situation evolved: SI ignored what 'contrarians' (the mainstream) said, the views they formed after reading SI's arguments, etc.
SI then gone to talk to GiveWell, and the presentation resulted in Holden forming same view - if you strip his statement down to bare bones he says that he thinks giving money to SI results in either no change, or increase of the risk, as the approach SI advocates is more dangerous than current direction, and the rationale given has already been available (but has been ignored).
Ultimately, it may be the case that SI arguments, when examined in depth by random outsider, typically result in strongly negative opinion of SI, but sometimes result in positive opinion of SI. The people whom form positive opinion seem to be a significant fraction at LW - ultimately if you examine the AI related arguments here, and form negative opinion, you'll be far less interested in trying to learn rationality from those people.
Is Holden's view really the same as the mainstream view, or is it just a surface similarity?
For example, a typical outsider would doubt about SIAI abilities, because a typical outsider thinks intelligent machines belong to sci-fi, not real life. Holden worries about lack of credentials. Among those who think intelligent machines are possible, a typical person thinks it will be OK, because obviously the machines will do only what we tell them to do. Holden worries that a (supposedly) Friendly AI is more risky than a "Tool AI". Etc.