5 Answers sorted by
40
Can other members of the household be questioned about their perception of the recipients' wellbeing before and after?
IMO it'd be ideal to ask them an open-ended "what changed about these people" to avoid priming effects.
However you choose to measure, be sure to check for any negative consequences of the donation. In adults, especially with mental health struggles, you may note resentment about how the aid was delivered or disappointment that it wasn't more.
It would be interesting to A/B test the impact of spending a lump sum like this, versus splitting the same sum over several weeks, months, quarters, or years. Part of experiencing a sense of security is knowing there'll be more where it came from -- so for instance it could be the case that giving a kid the interest from $1k as a monthly allowance might have greater positive overall impact than just giving them $1k worth of items all at once.
40
Do you have a good idea of what the situation looks like currently, what the case worker will buy, and what the situation looks like after it's been bought?
If you can extrapolate how the situation would look if you didn't make the intervention, and you know what the intervention did, you could try to compute the QALY difference between those two estimates to get an impression of its effectiveness. Remember to beware of regression to the mean, though.
Without this sort of detailed knowledge, I don't think it's possible to estimate the effect.
Those are all great questions. I’m going to start by asking the case worker those questions. I imagine they will buy items like diabetes medication, baby formula, and diapers. There my intervention frees up cash for the family to buy other items. Maybe I’ll direct the case worker to ask what they bought with the extra cash they have? I imagine the quality of life increase will come in the items they chose to buy with cash they didn’t have to spend on the necessities I bought.
3-2
It's going to be tricky. You may already be too close to the situation to judge impartially, and a case study is going to be difficult to use as evidence against population-level surveys of well-being, especially for your implied time horizon. You could attempt to benchmark against previous work, e.g. see what the literature has to say about the effects of poverty on diet, educational attainment, etc. in first-world cities, but your one new data point still won't generalize and it wouldn't be doing the heavy lifting in your argument for localism at that point.
I have never met the family and don’t plan on meeting them in person. A case worker will be in contact with them. I aim to maintain that distance, merely asking the case worker for data to plug into my spread sheet. The problem is that I do not know what data I should be plugging in.
If at all possible, I think it should really be recommended for you to meet them in person. Helps reduce problem due to Not Measuring What You Think You Are Measuring.
10
With a goal to "alleviate immediate and preventable suffering," QALY seems to be a pretty terrible metric. You need to measure immediacy, preventability, and suffering, or at least the suffering due to just the immediate and preventable causes. I would suggest suffering needs construct definition before you consider an operationalization.
It would be smart to measure pre- and post-intervention. The good news is that if suffering is a subjective psychological state, you could do post-only and measure perceived change. If you're worried about self-report, you could do observer-report (case worker), but since they'll be doling out the funds that could be biased as well (and presumably they'll be basing these decisions in part on self-reports of what is a "fire").
The type of counterfactual analysis tailcalled suggests is likely what the family or case worker would be mentally approximating when responding to how big of a difference paying off the utility bill or a pizza night made to their suffering. Plus, n=1, so a qualitative assessment may be all you can really do - if they rate something a 7/7 on reducing their suffering and everything else came in at 1-2, sure, that hits you between the eyes, though you'd probably get that anyway from them saying "the most important thing, by a mile, was X, and this is why..."
I am giving $5k to a case worker who will buy goods for a family in New Orleans. The mother and father of the family have schizophrenia and do not work. (The father is violent, and does not live with his children.) The mother and her children live in a two bedroom house with 10 other people. I don’t know the children’s ages, but I believe they are around 3, 7, and 12 years old.
The only directive I’ve given the case worker is to alleviate immediate and preventable suffering as best possible. “Put out fires,” so to speak.
I would like to scale this giving into a larger program of philanthropy if I can show that my $5k makes a positive impact. …This is where I get vague and need Rat/EA help… how do you think I should judge success? QALYs? How would I even judge QALY improvement?… In the end, I want to be able to say $5k to a family of 5 reduces suffering by a factor of X. I hope to compare these measures to EA-backed giving opportunities.
…Some additional background of why I’m doing this…
It’s easier to raise money for local causes that prevent suffering than something like the Against Malaria Foundation. You can see an example of this in the 11:11 group donations. A majority of the causes group members gave to were local causes, despite the entire group going through 11 weeks of EA indoctrination.
With proper rat/EA methodology, I hope to establish clear OKRs for a local charitable program. We would aim to fundraise to achieve these OKRs before diverting cash to other locals in need. (The ultimate goal is to raise money and reduce suffering in local communities, spreading excess resources from the wealthiest communities to the poorest communities.)
Curious to hear your ideas.