I'm also glad to see competition, and I would not be surprised at all if their overviews of the evidence were stronger than GiveWell's. It would be nice to see what they are doing, and I don't trust their results too much without that. I guess there is a book where they describe the meta-analyses they've done, which I have not had a chance to see.

The comparison with GiveWell seems mostly unreasonable, and I think reflects somewhat badly. Most of the points are either mistaken or misleading, and I would be surprised if they could be made by someone writing in good faith. [Edit: apologies for the snarky tone and inaccurate claim!]

• They suggest GiveWell doesn't understand causal attribution because Holden lists instrumental variables at the top of an inventory of methods of causal attribution. However, the list is in alphabetical order, and Holden says specifically that they rarely see compelling studies of this form.
• They criticize GiveWell for vote counting and provide a detailed description of what vote counting is. They link to GiveWell's analysis of microfinance as their only example, and suggest it makes elementary statistical errors. The procedure GiveWell is following is to look at a variety of studies with methodological failures, point out that the positive results are in the studies with the largest methodological failures, and that there is strong evidence that those failures will exaggerate the positive impact of microfinance. It is hard to mistake this for vote counting in good faith. A study which shows that eliminating methodological flaws reduces measured positive impacts does constitute evidence that undermines other methodologically flawed studies which find a positive effect. This is not the same as "accepting the null."
• AidGrade posts the comparison on their blog but doesn't take responsibility for it (disclaiming: these are the views of the author) or allow responses there.

Eva Vivalt thinks that making comparisons between outcome measures is not the place of a charity evaluator, and faults GiveWell for being willing to do so. No argument is provided for this, nor any response to the arguments GiveWell has given in favor of making such judgments.

This seems like a common and defensible position, but as an altruist concerned with aggregate welfare it doesn't make too much sense for me. Yes, there is value in producing the raw estimates of effectiveness on respective outcome measures (which GiveWell does as well), but encouraging discussion about what outcome measures are important is also a valuable public good, and certainly not an active disservice.

@Raemon: saying this is better for "people with choice paralysis or who don't have any idea how to evaluate different types of outcomes" seems to be missing the point. It is a significant, largely empirical challenge to determine which intermediate outcome measures most matter for the things we ultimately care about. Whether or not GiveWell does that passably, it is clearly something which needs to be done and which individual donors are not well-equipped to do.

The two valid points of criticism Eva raises:

• GiveWell is willing to accept data of the form "these two graphs look pretty similar" when common-sense reasoning suggests that similarity reflects a causal influence. At best, GiveWell's willingness to use such data requires donors to have more trust in them. At worst, it causes GiveWell to make mistakes. Such data was misleading once in the past, indeed in the example GiveWell cites of this approach. That said, a formal statistical analysis wouldn't have helped at all if GiveWell was willing to accept difference-of-difference estimators or matching [see the next point]. Overall this seems like a valid criticism of GiveWell, although I think GiveWell's position is defensible and has been defended, while this criticism includes no argument other than an appeal to incredulity. Indeed, even the incredulity is incorrectly described, as it applies verbatim to difference-of-difference estimators and matching as well, which economists do accept and which the author chastises GiveWell for not explicitly listing in their inventory.
• GiveWell fails to note difference-of-differences or matching in their list of methods of causal attribution. This is technically valid but a bit petty, given that GiveWell does in fact use such estimators when available. Making an error of omission in an expository blog post probably does not license the conclusion "[GiveWell is] not in a good position to evaluate studies that did not use randomization." [Edit: as Eva points out, GiveWell is in fact in a worse position to evaluate studies that don't use randomization, though I don't think the evidence presented is very relevant to this claim and I think Eva overstates it significantly.]