Follow up to: Why I'm Skeptical About Unproven Causes (And You Should Be Too)

-

My previous essay Why I'm Skeptical About Unproven Causes (And You Should Be Too) generated a lot of discussion here and on the Effective Altruist blog.  Some related questions that came up a lot was: what does it take to prove a cause?  What separates "proven" from "speculative" causes?  And how do you get a "speculative" cause to move into the "proven" column?  I've decided that this discussion is important enough that it merits a bit of elaboration at length, so I'm going to do that in this essay.

Proven Cause vs. Speculative Cause

My prime example of proven causes are GiveWell's top charities.  These organizations -- The Against Malaria Foundation (AMF), GiveDirectly, and Schistosomiasis Control Initiative (SCI) -- are rolling out programs that have been the target of significant scientific scrutiny.  For example, delivering long-lasting insecticide-treated anti-malaria nets (what AMF does) has been studied by 23 different randomized, controlled trials (RCTs).  GiveWell has also published thorough reviews of all three organizations (see reviews for AMF, GiveDirectly, and SCI).

On the other hand, a speculative cause is a cause where the case is made entirely by intuition and speculation, with zero scientific study.  For some of these causes, scientific study may even be impossible.

Now, I think 23 RCTs is a very high burden to meet.  Instead, we should recognize that being "proven" is not a binary yes or no, but rather a sliding scale.  Even AMF isn't proven -- there still are some areas of concern or potential weaknesses in the case for AMF. Likewise, other organizations working in the area, like Nothing But Nets, also are nearly as proven, but don't have key elements of transparency and track record to make myself confident enough.  And AMF is a lot more proven that GiveDirectly, which is potentially more proven than SCI given recent developments in deworming research.

Ideally, we'd take a Bayesian approach, where we have a certain prior estimate about how cost-effective the organization is, and then update our cost-effectiveness estimate based on additional evidence as it comes in.  For reasons I argued earlier and GiveWell has argued in "Why We Can't Take Expected Value Estimates Literally (Even When They're Unbiased)", "Maximizing Cost-Effectiveness Estimates via Critical Inquiry"</a>, "Some Considerations Against More Investment in Cost-Effectiveness Estimates", I think our prior estimate should be quite skeptical (i.e. expect cost-effectiveness to be not as good as AMF / much closer to average than naïvely estimated) until proven otherwise.

Right now, I consider AMF, GiveDirectly, and SCI to be the only sufficiently proven interventions, but I'm open to other organizations also entering this area.  Of course, this doesn't mean that all other organizations must be speculative -- instead there is a middle ground of organizations that are neither speculative or "sufficiently proven".

From Speculative to Proven

So how does a cause become proven?  Through more measurement.  I think this is best described through examples:

Vegan Outreach and The Humane League work to advertise people reducing the amount of meat in their diets in order to avoid cruelty in factory farms.  They do this through leafleting and Facebook ads.  Naïve cost-effectiveness estimates would guess that, even under rather pessimistic assumptions, this kind of advocacy is very cost-effective, perhaps around $0.02 to $65.92 to reduce one year of suffering on a factory farm.

But we can't be sure enough about this and I don't think this estimate is reliable.  But we can make it better with additional study.  I think that if we ran three or so more studies that were relatively independent (taking place in different areas and run by different researchers), addressed current problems with the studies (like lack of a control group), had longer time-frames and larger sample sizes, and still pointed toward a conversion rate of 1% or more, than I would start donating to this kind of outreach instead, believing it to be "sufficiently proven".

Another example could be 80,000 Hours, an organization that runs careers advice and encourages people to shoot for higher impact careers using their free careers advising and resources.  One could select a group of people that seem like good candidates for careers advice, give them all an initial survey asking them specific things about their current thoughts on careers, and then randomly accept or deny them to get careers advice.  Then follow up with everyone a year or two later and see what initial careers they ended up in, how they got the jobs, and for the group that got advising, how valuable in retrospect the advising was.  With continued follow up, one could measure the difference in expected impact between the two groups and figure out how good 80K is at careers advice.

Perhaps even The Machine Intelligence Research Institute (MIRI) could benefit from more measurement.  The trouble is that it's working on a problem (making sure that advanced artificial intelligence goes well for humanity) that's so distant, it's difficult to get feedback.  But they still potentially could assess the success or failures of their attempt to influence the AI community and they still could try to solicit more external reviews of their work from independent AI experts.  I'm not close enough to MIRI to know whether these would be good or bad ideas, but it seems plausible at first glance that even MIRI could be better measured.

And it wouldn't be too difficult to expand this to other areas.  For example, I think GiveWell's tracking of money moved is reliable enough and their commitment to self-evaluation (and external review) strong enough that I would strongly consider funding them before any of their top charities, if they ever had any room for more funding (which they currently do not and urge you to donate to their top charities instead).  Effective Animal Activism could do the same and I think have even higher success, because I think it's moderately likely that if someone starts donating to animal charities after joining EAA, there are few other things that could have influenced them.

Of course, these forms of measurement have their problems, and no measurement -- even two dozen RCTs -- will be perfect.  But some level of feedback and measurement is incredibly necessary to avoid our own biases and failures in naïve estimation.

The Proven and The Promising: My Current Donation Strategy

My current donation strategy is to separate organizations into three categories: proven, promising, and not promising.

Proven organizations are the ones that I talked about earlier -- AMF, GiveDirectly, and SCI.

Promising organizations are organizations I think have a hope of becoming proven, someday.  They're organizations practicing interventions that intuitively seem like they would have high upside (like 80K Hours in getting people into better careers and The Humane League in persuading a bunch of people to become vegetarian), have a good commitment to transparency and self-measurement (The Humane League shines here), and have opportunities for additional money to be converted into additional information on their impact.

My goal in donating would be to first ensure the survival of all promising organizations (make sure they have enough funding to stay around) and then try to buy information from promising organizations as much as I can.  For example, I'd be interested in funding more studies about vegetarian outreach or making sure 80K has the money they need to hire a new careers advisor.

Once these needs are met, I'll save a fair amount of my donation to meet future needs down the road.  But then, I'll spend some on proven organizations to (a) achieve impact, (b) continue the incentive for organizations to want to be proven, and (c) show public support for those organizations and donating in general.

...Now I just need to actually get some more money.

-

(This was also cross-posted on my blog.)

I'd like to thank Jonas Vollmer for having the critical conversation with me that inspired this piece.

On LessWrong, we often get cross, and then rude, with each other. Sometimes, someone then observes this rudeness is counterproductive.

Is it?

As a general rule, emotional responses are winning strategies (at least for your genes).  That's why you have those emotions.

Granted, insulting someone during your rebuttal of their argument makes it less likely that they will see your point. But it appears to be an effective tactic when carrying on an argument in public.

It's my impression that on LessWrong, a comment or a post written with a certain amount of disdain is more-likely to get voted up than a completely objective comment. A good way to obtain upvotes, if that is your goal, is to make other readers wish to identify with you and disassociate themselves from whomever you're arguing against.  A great many up-voted comments, including some of my own, suggest, subtly or not subtly, with or without evidence, that the person being responded to is ignorant or stupid.

The correct amount of derision appears be slight, and to depend on status. Someone with more status should be more rude. Retaliations against rudeness may really be retaliations for an attempt to claim high status.

What's the optimal response if someone says something especially rude to you?  Is a polite or a rude response to a rude comment more likely to be upvoted/downvoted?  Not ideally, but in reality.  I think, in general, when dealing with humans, responding to skillful rudeness, and especially humorous rudeness, with politeness, is a losing strategy.

My expectation is that rudeness is a better strategy for poor and unpopular arguments than for good or popular ones, because rudeness adds noise.  The lower a comment's expected karma, the ruder it should be.

You jerk.

AidGrade is a new charity evaluator that looks to be comparable to GiveWell. Their primary difference is that they *only* focus on how charities compare along particular measured outcomes (such as school attendance, birthrate, chance of opening a business, malaria), without making any effort to compare between types of charities. (This includes interesting results like "Conditional Cash Transfers and Deworming are better at improving attendance rates than scholarships")

GiveWell also does this, but designs their site to direct people towards their top charities. This is better for people with don't have the time to do the (fairly complex) work of comparing charities across domains, but AidGrade aims to be better for people that just want the raw data and the ability to form their own conclusions.

I haven't looked it enough to compare the quality of the two organizations' work, but I'm glad we finally have another organization, to encourage some competition and dialog about different approaches.

This is a fun page to play around with to get a feel for what they do:
http://www.aidgrade.org/compare-programs-by-outcome

And this is a blog post outlining their differences with GiveWell:
http://www.aidgrade.org/uncategorized/some-friendly-concerns-with-givewell

I read Optimal Employment and decided to try a work-holiday in Australia.  Several people have asked me how it's going, so here's my first take. I haven’t finished my work-holiday yet, but I want  to provide this information now so that potential travelers can make a decision before the hiring season peaks next summer.

Alice Springs

I came to Alice Springs in late April, and secured a job at a fast-food immediately. They offered me $26/hr after super and penalties. I didn’t get many hours, but it was enough to cover expenses. I took the job so that I wouldn’t lose money while I was looking for a better job.

Finding better job proved tough. I quickly applied to all of the sit-down restaurants and bars in town, and they all said they preferred locals. I signed up for a couple of employment agencies, but they were concerned that I had didn’t have any experience serving alcohol. I borrowed a phonebook and called random business on the Plenty Highway and that didn’t work.

This continued for five weeks. I found this period stressful because I get anxious asking businesses if they’re hiring, and then I feel humiliated when they say they’re not. There were fun times: I toured Uluru and met cool people in the hostel. But in the back of my mind I was always stressed about finding a better job. After five weeks I was about to head south in search of fruit picking work. But then I saw a flyer.

Remote Australia

A gas-station/convenience store was looking for staff. It was a full-time job, with meals and accommodation costing $40/week. This was exactly what I was looking for! I called immediately and got the job because I was American and nobody else had called before me.

This job was pretty much everything Louie described in Optimal Employment. I had practically no expenses, no responsibilities, and free entertainment after hours. At any given time there were three backpackers (including myself) working there, and one would be switched out about every month. We played a lot of video games and ate a lot of barbeques. We also attended a concert and a rodeo. I saved a lot of money and had a lot of fun, but I was anxious to leave after six months.

Adelaide

Australia prohibits backpackers from working the same job for more than six months, so I had to leave. Around this time I discovered that I had distant relatives living in Victoria, and they wanted me over for Christmas. This left me only three weeks to work. I decided to look for fruit picking work, since I heard they’re more willing to employ backpackers for short periods.

I flew to Adelaide, but I cannot find any fruit picking work. And I hear there's very little in Victoria. I could go to West Australia, but even if I found a job over there, it hardly seems worth it to fly there, work for two weeks, and fly back to Victoria on Christmas Eve. So now I’ve got to fill up two weeks in South Australia or Victoria.

What’s next?

I had originally planned to work 88 days of fruit picking work, so that I could get a second working holiday visa next year. (Most Americans are NOT eligible to do this. I can because I’m half British.) But I’ve heard a lot of bad things about fruit picking, that it’s miserable work and that you make little money (or even negative money). And I’m 70% sure I don’t want to come back to Australia next year anyway. (I want to teach English in Shanghai). So the new plan is to spend Christmas in Victoria while applying to TEFL jobs, and then tour the east coast for a month. Then I’ll fly back to the states and hopefully start teaching English in March. In the meantime I will have to entertain myself Adelaide/Melbourne.

Is Optimal Employment accurate?

Pretty much. You’ll probably earn more money on the job than Louie estimates, because of penalty rates and an increasing minimum wage. And there’s a chance to hit jackpot: I met a LW lurker in Alice Springs who said she was banking $1000/wk after expenses working at Lasseters. Mining and fishing jobs are also lucrative if you can get them.

But there’s also a risk of not finding a job. It took me five weeks to find my remote area job, and I’ve met some backpackers who ran out of money and while looking for work. Immigration requires you to save up $5000 before you start your work-holiday; skirt that law at your peril.

Overall I’m pleased with my visit to Australia. Even if I though I’ll be bleeding money the next couple of months, I’ll still come back with a nice profit and wicked memories. If you’re young with a sense of adventure, you should definitely consider it. A lot of people want to visit Australia “someday”, but it gets more difficult when you’re older and have more commitments.

I’m planning to write another post with tips for working in Australia. Requests welcome!

Just read this article, which describes a splashy, interesting narrative which jives nicely with my worldview. Which makes me suspicious.

http://dvice.com/archives/2012/10/ethiopian-kids.php

The One Laptop Per Child project started as a way of delivering technology and resources to schools in countries with little or no education infrastructure, using inexpensive computers to improve traditional curricula. What the OLPC Project has realized over the last five or six years, though, is that teaching kids stuff is really not that valuable. Yes, knowing all your state capitols how to spell "neighborhood" properly and whatnot isn't a bad thing, but memorizing facts and procedures isn't going to inspire kids to go out and learn by teaching themselves, which is the key to a good education. Instead, OLPC is trying to figure out a way to teach kids to learn, which is what this experiment is all about.

Rather than give out laptops (they're actually Motorola Zoom tablets plus solar chargers running custom software) to kids in schools with teachers, the OLPC Project decided to try something completely different: it delivered some boxes of tablets to two villages in Ethiopia, taped shut, with no instructions whatsoever. Just like, "hey kids, here's this box, you can open it if you want, see ya!"

Just to give you a sense of what these villages in Ethiopia are like, the kids (and most of the adults) there have never seen a word. No books, no newspapers, no street signs, no labels on packaged foods or goods. Nothing. And these villages aren't unique in that respect; there are many of them in Africa where the literacy rate is close to zero. So you might think that if you're going to give out fancy tablet computers, it would be helpful to have someone along to show these people how to use them, right?

But that's not what OLPC did. They just left the boxes there, sealed up, containing one tablet for every kid in each of the villages (nearly a thousand tablets in total), pre-loaded with a custom English-language operating system and SD cards with tracking software on them to record how the tablets were used. Here's how it went down, as related by OLPC founder Nicholas Negroponte at MIT Technology Review's EmTech conference last week:

"We left the boxes in the village. Closed. Taped shut. No instruction, no human being. I thought, the kids will play with the boxes! Within four minutes, one kid not only opened the box, but found the on/off switch. He'd never seen an on/off switch. He powered it up. Within five days, they were using 47 apps per child per day. Within two weeks, they were singing ABC songs [in English] in the village. And within five months, they had hacked Android. Some idiot in our organization or in the Media Lab had disabled the camera! And they figured out it had a camera, and they hacked Android.

So this sounds really inspiring and stuff, even subtracting some obviously sensational stuff (I assume "hacked Android" means "opened up the preferences dialog and flicked a switch"). I've poked around a bit and found similarly fluffy pop-philanthropy articles. Anyone know if there's more reliable information about this out there?

THINK, The High Impact Network, is going live this week.

We're a network of Effective Altruists (EAs), looking to do the most good for the most people¹ as efficiently as possible. We aren't bound by a central cause or ethical framework, but rather by a process, and a commitment to rigor and rationality as we try to make the world a better place.

THINK meetups are forming around the world. Some are functioning as student groups at prominent universities, others are general meetups for people of all ages who want to make effective altruism a part of their life. As I write this, 20 meetups are getting ready to launch in the fall, and discussions are underway for an additional 30. If you'd like to connect with other EA-types, see if a meetup's forming in your area, or run your own meetup, send us an e-mail here, or visit our website.

We're putting together a collection of meetup modules, which newly formed groups can use for content at weekly meetups. These fall into roughly two categories:

• Introductory materials, designed to teach the basics of Effective Altruism to newcomers.
• Self Improvement tools, helping newcomers and veterans to become strong enough to tackle the difficult problems ahead.

Five sample modules are available on our website, and more are coming. If you have ideas for a module and would like to create you own, e-mail us at modules@thehighimpactnetwork.org.

But most importantly - we want bright, enthusiastic people who care deeply about the world to collaborate with each other on high impact projects. 

Optimal Philanthropy. Effective Altruism.

Less Wrong veterans will recognize the basics of Optimal Philanthropy, although we consider avenues beyond traditional charity. (The phrase "effective altruism" was settled on after much deliberation). For those unfamiliar, a brief overview.

Over the past decade, important changes have begun to take root in the philanthropy/altruist sector:

• Organizations like Givewell, as well as a growing number of foundations like the Gates Foundation, are shifting the discussion of giving towards efficiency and evidence.
• Groups like Giving What We Can and Bolder Giving are encouraging people to incorporate philanthropy into their lifestyle. You can donate 10% or more of your income and still be among the richest people on the planet, living a satisfying life.
• The organization 80,000 Hours is promoting high impact career choice. You'll spent thousands of hours at your job. You can accomplish dramatically more good for the world if you optimize for it.

Above all, serious discussion is slowly mounting towards an incredibly important question - if you want to have the biggest impact you possibly can, what do you do?

Donating to provably efficient charities is an obvious first step, but more is possible. Systemic changes can have a powerful impact. New technologies have the potential to radically improve lives - as well as the capacity to destroy life as we know it. The Singularity Institute, the Future of Humanity Institute, Givewell and others are all in the process of grappling with this problem. I think it's fair to say that the Less Wrong community has had a noteworthy impact on the discussion.

Among the most valuable things the Less Wrong community has taught is the importance of... well, community. For Effective Altruism to be successful as a movement and a lifestyle, it needs people working together who share a passion for it, a commitment to intellectual rigor, and a sense of humor. People who can help each other grow, collaborate on important projects, and more.

THINK. The High Impact Network. Ready to launch this fall.

¹ _{Where by "help 'people'" we mean "and animals too." Depending on your ethical framework. Probably not including clams. Quite possibly including future sentient beings of various sorts. It's complicated. Come to a meetup, we'll talk about it.}

I managed to turn an essay assignment into an opportunity to write about the Singularity, and I thought I'd turn to LW for feedback on the paper. The paper is about Thomas Pogge, a German philosopher who works on institutional efforts to end poverty and is a pledger for Giving What We Can. 

I offer a basic argument that he and other poverty activists should work on creating a positive Singularity, sampling liberally from well-known Less Wrong arguments. It's more academic than I would prefer, and it includes some loose talk of 'duties' (which bothers me), but for its goals, these things shouldn't be a huge problem. But maybe they are - I want to know that too.

I've already turned the assignment in, but when I make a better version, I'll send the paper to Pogge himself. I'd like to see if I can successfully introduce him to these ideas. My one conversation with him indicates that he would be open to actually changing his mind. He's clearly thought deeply about how to do good, and may simply have not been exposed to the idea of the Singularity yet.

I want feedback on all aspects of the paper  - style, argumentation, clarity. Be as constructively cruel as I know only you can.

If anyone's up for it, fee free to add feedback using Track Changes and email me a copy - mjcurzi[at]wustl.edu. I obviously welcome comments on the thread as well.

You can read the paper here in various formats.

Upvotes for all. Thank you!