Another possibility is that people really don't like the idea of having to do a 2 to 4 hour deliberately useless test assignment.
It's a tempting hypothesis, but I've heard enough complaints about volunteers from organizations that don't do that sort of test that I don't think the test is a crucial factor.
Identifying conscientious people seems to be a hard problem. Do you have suggestions about how it can be done?
I posted to this thread basically to add my complaint. I am involved with a not-for-profit and there is a huge problem with people who enthusiastically volunteer but end up doing little or nothing of value for the organization.
So yeah, I don't think it's that people find opportunities elsewhere or that people resent make-work.
Here's a thought experiment: Suppose that Givewell continues to test volunteers, but instead of something boring and tedious, the "test" is something fun and interesting. But still it is admittedly make-work. I predict that the percentage of applicants who complete the test will rise dramatically.
I presume that the reason they put this program in place and began to keep statistics is that they perceived a significant issue with volunteers never completing assignments. Yes it would be nice if we had more data on that, but without that data it would be really hard to tease out the significance of this marginal difference.
Volunteers who fail to complete real assignments can be very disruptive to an organization and a drain on its other resources who must put in time on training, review and various forms of hand-holding. So, it is probably not a bad idea for Givewell to do this. Naturally volunteers do not want to spend several hours on a make-work assignment but if they are truly interested in benefiting this particular organization then they should be able to see the larger picture that makes it necessary.
The characterization of the test assignment as deliberately useless strikes me as unfair; formatting the lists of sources is useful even if marginally so.
there is likely low-hanging fruit for volunteers in charities or communities in the area of sustained tedious tasks; collecting anecdotes, reports, links, that sort of thing come to mind as LW examples
It seems like most of this can already be accomplished very cheaply by Mechanical Turk.
I'm sure, but from reading the Mechanical Turk literature (eg. my boss is a robot), you need really specific tasks and a framework for working through MT so you can do majority voting on submissions and that sort of thing.
I have a hard time thinking of things you would want to do that are mechanical enough to get good results out of Turk, and large-scale enough that you would recover all the overhead of using MT and the actual fees (to say nothing of learning how to use MT and your framework!).
The article resonated with me because a number of my own activities are pretty repetitive volunteer stuff; like reading through the archives of the Evangelion ML looking for forgotten gems and information, but I can't imagine trusting that to Turkers because so many of the important parts are things I can't explain and only recognize their importance serendipitously and sometimes only in retrospect after I have learned about a related topic. Or writing the DNB FAQ, something which any volunteer could do if they just read through the DNB ML's emails to see what is important and consolidated it all, but again not something you could really write explicit instructions for.
Another option might be something less mechanized like freelancer.com (which another LW recommended as useful). Do you see that as more viable?
That'd probably work much better as long as you are able to pick a freelancer with skills or credentials matching your needs. Still wouldn't help with my Evangelion example, but would with the DNB FAQ (pick someone with Internet and psychology skills and tell them 'flag any email that looks interesting for an FAQ etc.').
On the other hand, you would also be paying even more on freelance or elance, and at that point, you're not really dealing with volunteer labor but lightweight outsourcing. (I don't know if freelancer.com even lets you pay $0 for volunteers, like Mechanical Turk does.)
I have a hypothesis, which is that a number of those people found somewhere else to spend their hours in the meantime (Possibly becoming valuable volunteers at other locations) and simply forgot to inform Givewell.
As an example of how this might work, I'll imagine a scenario where I apply to do something at 125 organizations. I hear back from 25 of them. I respond to 5 interviews or tests. An organization and I eventually select 1 final place to spend my time at.
This would be a circumstance where there is only one organization which thinks "Ah, a valuable worker." out of 125 organizations who saw my application. It leaves also leaves a number of loose ends where people can drop off.
Although, as evidence against this hypothesis, it seems a bit odd that not one person Givewell tracked had a response along the lines of "Thank you for your interest, but I have found another opportunity elsewhere." I mean, if my hypothesis happened to be the case, it seems like I should expect at least a one person to have done this out of the 27 who applied but did not end up doing valuable work.
I can't test the hypothesis myself, but Givewell could always send out a followup survey if they wanted to test the idea.
Possible solution, avoid make-work: http://lesswrong.com/r/discussion/lw/c4c/experiment_a_good_researcher_is_hard_to_find/
From the GiveWell blog, which is often interesting & applicable to our interests, comes a post on the quality of their volunteers:
(The dropout rate is probably not due to the perceived low utility of the work - GiveWell seems to be up-front that the test assignment is a test.)
I draw a few lessons from this: