You don't need to come up with hypothetical difficult-to-measure charities. As I've noted before, the WIkimedia Foundation is a tricky one. We do want your money, because it takes money to run, but for the current funding drive we've actually deliberately chosen a strategy that maximises the number of donors, to protect the projects' editorial independence (not to be potentially beholden to major donors).

Pretty much all of the valued output (the projects) is not done by the staff, but by volunteers who write stuff and take photos and so forth. This already made our GuideStar rating difficult, for instance.

And how do we measure? What do we choose to measure, knowing that we'll just get what we measure? Everyone has an opinion on what they want out of Wikipedia, for example, but how do we assess the CEV of our users in a way that we can measure the results of then acting upon and not just get variants on "more of the same please"? At present we're launching a quite speculative programme to get stuff happening on the wikis in India, for example. It's a defensibly good idea, but what measurement system would come up with that goal?

There must surely be other extant charities that are quite difficult to quantify results per dollar. jsalvatier suggests open source software projects, for example. Your money is somewhat useful, but your time is far more useful.

A huge problem with this approach is that your intuition is really bad at judging such approaches, for several reasons. 1) your situation is vastly different than those of the people you're trying to help 2) unlike almost everything else you evaluate in life, you get no feedback unless you specifically seek it out 3) the effects you're trying to judge are far in the future. Given that there are many more ways to waste resources than to do something effective, this means that unless you have very strong evidence that what you're doing is effective, you're almost surely simply wasting resources by focusing on such approaches.

My think my ideal charity would be something similar to Heifer International, but which also focuses on reproductive health and/or women's rights. Feeding people fish for a day means you just need to feed them again tomorrow, and if they have a bunch of kids you haven't necessarily accomplished anything. From what I've read, in places where the standard of living improves and women get more equality, overpopulation becomes less of an issue. So it seems to me that addressing those issues together in particular regions would produce sustainable longterm benefit. But Givewell doesn't seem to have a lot of information on those types of charities.

I'm sympathetic and had precisely the same initial reaction to GiveWell's recommendations when I encountered them a little more than a year ago.

Encouragingly, as Unnamed mentioned, some charities that improve health in the developing world may have long term positive impact.

Unfortunately, I haven't yet encountered programs with encouraging evidence of impact that aim to improve reproductive health and/or women's rights directly. My rough impression is that there seem to be serious difficulties engaging with such complex issues as an outsider to the relevant cultures. This is in line with jsalvatier's comment.

At the same time, there's more research to be done on this point. My impression is that GiveWell does plan to cover more international causes over time. For the time being they've written a Do-It-Yourself charity evaluation guide for donors interested in causes that they haven't yet covered.