Unnamed comments on Many Weak Arguments vs. One Relatively Strong Argument - Less Wrong

20 Post author: JonahSinick 04 June 2013 03:32AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (86)

Comments (86)

You are viewing a single comment's thread.

Comment author: Unnamed 04 June 2013 08:58:11AM 18 points [-]

Let's give this a try.

Claim: Relying on few strong arguments is more reliable than relying on many weak arguments.

  • Motivated reasoning is a bigger risk when dealing with weak arguments, since it is relatively easy to come up with weak arguments on the side that you favor, but it is hard to make an argument rigorous just because you want it to be true. It also seems easier to ignore various weak arguments on the other side (or dismiss them as not even worth considering) than to dismiss a strong argument on the other side.
  • Selection effects will tend to expose you to more weak arguments on one side of an issue; e.g. if you are surrounded by Blues then you will be exposed to lots of weak arguments in favor of Blue positions, and few arguments in favor of Green positions. A person in this Blue-slanted situation has a better chance of finding their way into the pro-Green camp on an issue if they ignore the argument count and instead only compare the strongest pro-Blue argument that they have seen with the strongest pro-Green argument that they have seen (or, even better, the steel-manned version).
  • The 80/20 rule: in many domains, a small fraction of the things carry a large portion of the weight, and a useful heuristic is to focus on that small fraction (e.g., the 20% of effort that produces 80% of the results). Which suggests that, in this domain, the strongest few arguments will carry most of the evidential weight on an issue, and the long tail of weak arguments will not matter much.
  • Nonindependence: a set of arguments on a given issue are rarely independent; arguments which share a conclusion often have strong (and perhaps hidden) dependencies and interrelationships. For example, a large fraction of the set of arguments may all rely on the same methodology, or come from the same group of people, or be (perhaps indirect) consequences of a single piece of evidence, or share a single auxiliary assumption. So a set of seemingly independent arguments often provides less evidence than it appears.
  • Argument structure: the structure of a complex argument is often important but neglected, and it is not accounted for by listing simple points in favor of each side. To take one example, the claim IF (A or B or C or D or E) THEN Z has a very different structure from the claim IFF (A & B & C & D & E) THEN Z, but moderate evidence against D would appear similarly as "a weak argument against the claim" in both cases. Making a strong argument requires engaging with the structure of the argument.

I can already see some counters to these arguments (and some counters to those counters), but I suspect it would be more useful to have a list of arguments on the other side in the same format to compare these with.

Comment author: JonahSinick 04 June 2013 03:39:58PM *  6 points [-]

Thanks for these thoughts.

Broadly, my reaction is that there's no royal road to rationality: one has to make judgments on a case by case basis. I haven't shifted over to using many weak arguments rather than a few strong ones in all instances.

If nothing else, my post shows that:

  • It's possible to justifiably have high confidence in a position based on many weak arguments, when there are no strong arguments on the other side.

  • I was making the mistake of completely ignoring certain pieces of weak evidence when I should have been giving them some weight.

Comment author: ThisSpaceAvailable 09 June 2013 06:50:45PM 1 point [-]

It's very difficult to come up with truly independent arguments. The examples given aren't even close. WA1 is that people majoring in a quantitative subject in general have higher earnings. WA2 is that a specific subsets of jobs that require a quantitative subject major have high earnings. Clearly, these are not independent.

Comment author: JonahSinick 07 June 2013 11:00:47PM 0 points [-]

I replied here.

Comment author: Kawoomba 04 June 2013 03:22:49PM 0 points [-]

Nonindependence (...) For example, (...) come from the same group of people

That is not nonindependent, i.e., if that were nonindependent, a human being would be incapable of ever giving independent arguments.

Comment author: Unnamed 04 June 2013 06:37:09PM 2 points [-]

The kinds of examples I had in mind with that phrase: 1) a bunch studies have been published which each provide some support for claim X, from a variety of different angles, but they were almost all conducted by the same group of 4 researchers. 2) You don't know much about nutrition and then read a book by Gary Taubes; now you have a lot of arguments in favor of low carb diets.

The general pattern here is that the object-level evidence (e.g., the findings of each particular study, or the content of each particular Taubes argument) does not entirely screen off the source. There are various pieces of information which you could potentially learn about the 4 researchers or about Taubes which would weaken your confidence in the whole set of arguments.

Comment author: Decius 04 June 2013 09:07:13PM -1 points [-]

Better claim: "In the absence of a coherent strong argument, the consideration of many weak arguments is expected to tend toward accurate conclusions."

moderate evidence against D would appear similarly as "a weak argument against [z]" [given] "IFF (A & B & C & D & E) THEN Z"

Wrong. Moderate evidence against D is moderate evidence against (A & B & C & D & E).

Powered by Reddit Powered by Reddit