All of Michael Simm's Comments + Replies

We need to take a bit of a step back here. I am just as keen as you are to get good unbiased data that can be relied upon to know precisely how impactful cash transfers - and other interventions - are at helping people. And I'd like to acknowledge that you're correct that we should not rely solely on unchecked survey data when trying to figure out impact results.

So I did a bit of a deep dive into meta-analyses and systematic reviews of cash transfer studies. It turns out that while surveys are generally one part of the data collected, researchers have been... (read more)

Our organization is not big enough to hire a statistician, although we will for sure get one when we are able to build a sufficiently large study / program. I'd be happy to refer you to the people that do have a ton of statisticians: 

https://www.givedirectly.org/research-at-give-directly/

https://basicincome.stanford.edu/research/ubi-visualization/

https://www.penncgir.org/research

Let's use a different analogy. Let's say that you are in exactly the same situation you are in right now, and some random organization decided to start giving you $1,000 check... (read more)

Good thing I'm not particularly interested in being a charity, I'm interested in building a tool for funders and fundraisers that maximizes their impact against poverty. So bring on the criticism.

That said, I think you're being excessively negative towards surveys in general. There are two primary reason why surveys are (seemingly) the only way to go about getting decent data, and not nearly as unreliable as you're suggesting.

  1. I can't think of another way to go about collecting more accurate & less biased information. The only way 100% know exactly what
... (read more)
2Lao Mein
You're the one with the novel hypothesis here. The burden of proof is on you. And I am rightfully suspicious of survey results and believe survey-only studies are of almost no use. Why? Because self-report is fundamentally unreliable. Yes, I am absolutely saying that people who receive money from you are incentivized to lie to you out of a sense of social obligation. Yes, I am saying that this means any study where you provide help and then rely on self-report as the only form of feedback is borderline useless and will convince few people. And no, it's not an unfalsifiable idea - objective metrics can be used as a comparison. If it is impossible to collect such data ethically, then you should stop doing your studies immediately. If your best piece of evidence in support of your ideas is just "think about it logically", I would highly recommend that EA charities never fund this project. To avoid embarrassment and awkwardness. To make the researchers feel better. This is a well-known problem in psychology called the "self-report bias", which is why good psychological studies never rely on self-report alone. As verification, you can also interview other family members, and employers, check arrest records, ect. If there is no way for you to do this ethically, then what do you think your studies are achieving? You ask someone if it's OK for you to drop by at their place sometime, and for their address. This is verifiable information, and they aren't likely to lie to you. If you ask them about things they know you can't check, they are much more likely to make a socially desirable lie. They know you can't check how much money they're spending on drugs, or if they're employed. People even lie to themselves about how much they exercise all the time. I'm not sure what your position here is. That people don't make prosocial lies out of a sense of obligation? People lie about how good each others' cooking is to avoid hurt feelings. Most explanations of the self-report bias

I do not agree that hospital visits and incarceration statistics, (although I'd love to have those numbers), are foundational to measuring the impactfulness of a homeless intervention:

Overwhelming demographic data as well as medical analysis make it evident that living on the streets directly accounts for most, though not all, of the massive mortality rate increase. There is a causal relationship between living on the streets and high death rates, especially in Arizona due to the high summer heat.

I'd be happy to hear why you might disagree, but I beli... (read more)

2Lao Mein
The only feedback you are getting is from surveys. They are subjective by definition. Gifts provide immense psychological pressure for reciprocation, especially if it's presented as "no-strings-attached". Every metric you cite improvement in is subject to this. This is why I'm so troubled by the lack of objective metrics or any other attempt to mitigate this. The fact that this isn't addressed is a major red flag. Do you have any statisticians associated with the project to help with study design?  Let me give an analogy. Let's say you dropped out of college and play video games all day. Your parents call in on the weekends and ask you about your job search and how much video games you're playing. Do you think you're more likely to lie or exagerate about your achievements if they're paying your rent? Even if you knew for sure they weren't going to pull finiancial support regardless? And what if this continued over the course of months?  I will write up an article some time this week regarding all this.

Those people are trying to persuade the whole public at first and then moving. With this apporoach we first move and then show it was a good thing. Sure need to get funders on board but private money pushing ahead of public policy is a "shoot first, ask permission later" approach.

You're pretty much right. I started my journey writing about UBI policy and its potential to improve society, but I got fed up with politics. I do think the danger isn't quite as bad at first. We basically have permission to 'shoot' because guaranteed income pilots are very common... (read more)

Thanks for reading this 30-minute thing. I first wanted to make a short 5-minute read but I realized that many of you would probably really want all of the evidence laid out clearly, and our plan explained in excruciating detail. - so you can point out the super obvious reason why this has a 0% chance of success, that I've completely overlooked  -  despite my search for fundamental issues since I came up with the idea, and asking all of the experts I can find

The EA community is probably the most knowledgeable community in the world about h... (read more)

Taking out at scale the most extractive labour globally will both do a lot of good and draw the ire of the most aggressive economic players. Making a chair taller by chopping of a leg for building materials is not a repeatable strategy. One may not like what is going on in the kitchen, but serving unprocessed ingredients will improve nutrition and will drive down patreonage.

I think I'm missing your point. UBI is a long way off, but there are a lot of (mostly economic theory) writings about how guaranteed income at a large scale would drastically shift powe... (read more)

7Slider
Those people are trying to persuade the whole public at first and then moving. With this apporoach we first move and then show it was a good thing. Sure need to get funders on board but private money pushing ahead of public policy is a "shoot first, ask permission later" approach. Climate change people need to deal with misinformation and mudding of the waters by oil companies and such. It is not just that the public randomly happens to be ignorant and a simple informing will do the trick. There is a lot of PR done by neoliberalism and such which will reactively up its efforts when the boat starts to shake. The pipedreams talk about the future but the grassroots squashing is done today. Western intelligence agencies, atleast for the bad apples part, will topple foreign orders if the environment is not sufficiently corporation-friendly. If the issues become hot, freedom of operation or experimentation might not be there. And here scale matters, a political wind that 1% of the population is symphatetic to can be tolerated. So lifting individual people up does not pose a threat to the way of life. But doing that to whole classes of people does more than just the sum of the individual cases. Sure more things become possible but pushing an old system aside means everybody invested in the old way will have their survival instincts triggered. You don't want to be on the business end of capitalism defending itself. So knowing which structures are (felt) existentially central is key to letting sleeping bears lay. Say you have a 20 M program running in a country. Yay, people are free to self-improve. But it also means they are not doing their previous income activity. Will there be a search for all those task that these people were doing? Probably. And because they were the bottom 20 M the replacements will not do it for cheaper (this is somewhat unique condition for this setup (normal economic thinking assumes that others step up)). Sure some tasks are not worth the new c

First, thank you for your rigor in analyzing the homelessness part of the post! I most certainly agree with you that cash transfers - explicitly relating to homeless individuals - need more studies and more rigorous RTCs from independent sources. 

According to surveys given to the participants. Even if you tell them that the payments aren't contingent on positive results, they don't necessarily believe you. And even if they do, they'll still feel obliged to give you the results you're after. This is a commonly known effect in psychology and sociology,

... (read more)
1Lao Mein
The fundemental issue of these studies is the reliance on surveys. In most of them, they are the only source of feedback! Why wasn't there any integration of objective measures like incarceration and hospital visits from the start? This isn't a minor issue - it's a flaw in the foundation.

I think at first, definitely not. I see it playing out like this: 

  1. While other forms of assistance are going about things like normal, someone (us) builds a big enough guaranteed income program to provide half or most of the homeless population in an area with guaranteed income.
  2. When that program happens, hopefully, most homeless people attain far better situations within a year, and the existing assistance services find themselves with more resources available to assist fewer people (the ones in highly bad mental states/addiction).
  3. Using guaranteed inco
... (read more)

So we've got two major types of guaranteed income experiments, those on homeless individuals, and those on the general (impoverished) public.

I agree with you that the New Leaf Project experiment (that I cited quite a bit) was quite positive, and also that the problem is still getting worse. Although It's important to note that that experiment only had 115 participants, so it covered only a drop in the bucket. 

The real question is, "If guaranteed income was scaled up to cover the entire population in homelessness or in danger of becoming homeless, woul... (read more)

2Mitchell_Porter
Is it intended as a substitute for other forms of assistance?

to be successful one would need to guarantee UBI to everyone, without means testing, for credibly unlimited duration, linked to a reasonable index to protect against inflation.

I would like to see a reason stated for any of these assertions. We're not doing UBI, or unlimited durations (at first at least). We also will be forced to do some means testing because of IRS limitations on 501(c)(3)s, but it will be only a 30-minute application. Our initial experiments will not be indexed to inflation because we still need to figure out the optimal amount to help p... (read more)

4Shmi
Hmm, you are certainly infinitely more familiar with the topic. I basically followed Scott Alexander's blog posts about UBI which is not what you are trying to do here. I agree that giving money directly makes more sense than most alternatives. I am guessing that there is a subset of the population for whom it will work, and won't just go into, say, drugs and gambling without a noticeable change in quality of life or mortality, but identifying it might be tricky, so you have to accept that a fair amount of money does not do any good. There was a local experiment with a one-time transfer that had some positive results https://forsocialchange.org/new-leaf-project-overview but the homeless situation here is getting progressively worse still.

Thanks for the comment! I also do not think cash will be more effective for - every - impoverished person. I do think, however, that most are a lot closer to almost all with the only exceptions being people that are suffering from extreme drug addiction and/or mental illness. Actually, there's some spotty evidence (we need a good RTC study on this) that cash transfers could actually be a cost-effective way to reduce drug usage. (check out Simon, a case study who was a 20-year homeless heroin addict).

individuals come and go from populations, and there's a t

... (read more)