This is a linkpost for https://forum.effectivealtruism.org/posts/NgSfKWayWpLwn8Lz7/lifeguards-1
You try to grab their attention—you scream at the lifeguards. But they don’t move.
At this point one ought to be surprised enough to doubt their own reasoning. Maybe even realize they are dreaming.
"Oh man! I'm a hypothetical character in one of those moral thought experiments! Even if I jump in and save the child, both they and I are going to vanish in a puff of logic once the scenario is resolved. :-( :-( :-( "
1
Many readers will be familiar with Peter Singer’s Drowning Child experiment:
Olivia Jimenez recently thought of a new version of the thought experiment.
2
People often ask us for advice as they consider next steps in their careers. Sometimes, we suggest ambitious things that go beyond someone’s default action space.
For example, we might ask Alice, a college junior, if she has considered trying to solve the alignment problem from first principles, found a new organization in an area unrelated to her major, or do community-building in a part of the world that she hasn’t visited.
Alice responds, Wait, why would I do that? There must be people way more qualified than me—isn’t this their responsibility?
---
The issue is not that Alice has decided against any of these options. Any of these options might be an awful fit. The issue is that she doesn’t seriously consider them. The issue is she has not given herself permission to seriously evaluate them. She assumes that someone else more qualified than her is going to do it.
In EA, that’s often just not the case. It’s good to check, of course. Sometimes, there are competent teams who are taking care of things.
But sometimes, there is not an imaginary team with years of experience that is coming to save us. There is just us. We either build the refuges, or they don’t get built. We either figure out how to align the AI, or we don’t.
Sometimes, there is no plan that works. Sometimes, there is no standard job or internship that you can slot into to ensure a bright future.
3
Before jumping into the pond, you should be mindful of some caveats.
Sometimes, there are competent lifeguards. There are people more capable than you who will do the job.
Sometimes, there is a good reason why the lifeguards aren’t intervening. Maybe the child isn’t actually drowning. Or maybe jumping into the pond would make the child drown faster.
Sometimes, jumping in means other lifeguards are less likely to dive in. They might assume that you have the situation under control. They might not want to step on your toes.
Try to notice these situations. Be mindful of alternative hypotheses for why no one seems to be jumping in. And be on the lookout for ways to mitigate the downsides.
But sometimes, the child is drowning, and the lifeguards aren’t going to save them.
Either you will save them, or they will drown.