Push-polling (a survey that purports to collect information, but is actually meant to change the respondents' behavior) is very clearly in the dark arts. It probably has a place in the world, but it's not about rationality, it's about manipulation.
Causing people to change their behaviour to your favourite behaviour by means other than adding safe input to people's rational deliberation processes seems questionable. Causing people to learn more about the world and give them the opportunity to change their behaviour if they feel it's warranted by the state of the world seems good. This post seems like it's proposing the latter to me - if you disagree, could you point out why?
I admit a "coordination survey" sounds like it's asking "who is going to do this (giving people the option of signing themselves up for something)" or "how are we going to do this" (what's the next step) or "how much would we have to pay you to do this"/"how much are you willing to pay someone to do this"(Pairing people up/Patreon/kickstarter).
I think if it's clear up front that you're not asking for "information" but asking "who wants to do this?" then that isn't being manipulative - provided the survey is appropriately marked "coordination", or "signing up for X".
Someone could do a survey to see how people feel about it, and/or what alternate names they would like better, which could be iterated into the next version of the survey.
On reflection, I endorse the conclusion and arguments in this post. I also like that it's short and direct. Stylistically, it argues for a behavior change among LessWrong readers who sometimes make surveys, rather than being targeted at general LessWrong readers. In particular, the post doesn't spend much time or space building interest about surveys or taking a circumspect view of them. For this reason, I might suggest a change to the original post to add something to the top like "Target audience: LessWrong readers who often or occasionally make formal or informal surveys about the future of tech; Epistemic status: action-oriented; recommends behavior changes." It might be nice to have a longer version of the post that takes a more circumspect view of surveys and coordination surveys, that is more optimized for interestingness to general LessWrong readers, and that is less focused on recommending a change of behavior to a specific subset of readers. I wouldn't want this shorter more direct version to be fully replaced by the longer more broadly interesting version, though, because I'm still glad to have a short and sweet statement somewhere that just directly and publically explains the recommended behavior change.
tl;dr: If this post included a section discussing push-poll concerns and advocating (at least) caution and (preferably) a policy that'd be robust against human foibles, I'd be interested in having this post in the 2019 Review Book.
I think this is an interesting idea that should likely get experimented with.
A thing I was worried about when this first came out, and still worried about, is the blurriness between "survey as tool to gather data" and "survey as tool to cause action in the respondent."
Some commenters said "this seems like push-polling, isn't that bad?".
I think a carefully executed version of this wouldn't be push-polling. I'm not 100% sure whether the example here counts or not. It doesn't prescribe an action. But it does prescribe a frame. There's a spectrum of push-polliness, and I say the example here is non-zero push-poll-y, even if not maximally.
I don't think it's intrinsically bad to be push-poll-y, but I think it's useful to draw distinctions between surveys designed for gathering information vs shaping actions. If a tool like this started seeing common use, we'd a) start seeing some variants of it that were push-poll-y (because humans aren't always careful), and b) I think people receiving the survey would be anxious about it being push-poll-y, which would compromise their trust in future surveys on the same topic or from the same institutions. This may be unfair, but it's the outcome I predict.
One commenter suggested flagging such a survey as a "coordination survey" rather than a "information gathering survey." This is somewhat unfair to hypothetical survey-makers who were very careful to not be a push poll. But, I think it is probably the best equilibrium.
(Flagging polls as "coordination surveys" will probably come with some consequences, where people who are already mistrustful of your cause and don't want to coordinate on it are more likely to avoid the survey in the first place. But the alternative is manipulating them into taking a survey that wasn't quite what they thought it was. I think that is worse)
I think something like this might be pretty valuable among EAs/rationalists dealing with COVID. We clearly have a lot of people doing independent research, and a lot of it might be redundant.
I think EA/rats are mostly patting ourselves on the back RE how well we predicted COVID, and not thinking about how we could've done even better. I think we could've done better at coordinating within these communities, e.g. in an ideal world, we might've done better (and might do better) by coordinating efforts to channel COVID-efforts into producing community-facing or public-facing reports / recommendations. And we might've empowered some community members to be our "COVID czars" (e.g. by a combination of volunteering, voting, predicting who would do a good job, and funding them), and thus allowed many community members to spend a lot less energy on COVID.
Yeah, I've been thinking the same. It feels like there are a number of action-coordination dimension where we could have done substantially better (a substantial number of which will still be relevant for a while, so there is still time to improve).
Planned summary:
This post suggests that when surveying researchers about the future impact of their technology, we should specifically ask them about their beliefs about what actions other people will take, and what they personally are going to do, rather than just predicting total impact. (For example, we could ask how many people will invest in safety.) Then, by aggregating across survey respondents, we can see whether or not the researchers beliefs about what others will do match the empirical distribution of what researchers are planning to do. This can help mitigate the effect where everyone thinks that everyone else will deal with a problem, and the effect where everyone tries to solve a problem because they all think no one else is planning to solve it. Critch has offered to provide suggestions on including this methodology in any upcoming surveys; see the post for details.
Planned opinion:
This is a cool idea, and seems worth doing to me. I especially like that the survey would simply reveal problems by collecting two sources of information from people and checking their consistency with each other: there isn't any particular argument being made; you are simply showing inconsistency in people's own beliefs to them, if and only if such inconsistency exists. In practice, I'm sure there will be complications -- for example, perhaps the set of researchers taking the survey is different from the set of "others" whose actions and beliefs they are predicting -- but it still seems worth at least trying out.
This post introduced to me a simple way to use surveys to build common knowledge that naturally leads to action. I still feel excited about it and I think about it every time I consider making a survey.
I agree it would be good to add a note about push polling, but it's also good to note that the absence of information is itself a choice! The most spare possible survey is not necessarily the most informative. The question of what is a neutral framing is a tricky one, and a question about the future that deliberate does not draw attention to responsibilities is not necessarily less push-poll-y than one that does.
I was kind of on the edge on whether to nominate this post, but David Krueger's point convinced me that maybe if I had remembered this post more, we would have done quite a bit better on COVID, and I do think that seems really quite important.
It seems you mismatched the names in this section
Alex: “Why worry about AI safety? It would be silly to make AI unsafe. Therefore someone will take responsibility for it.”
Bailey: “You should definitely worry about AI safety, because many people are not taking responsibility for it.”
These views are strangely compatible and therefore hard to reconcile by evidence alone. Specifically, Alice is rightly predicting that people like Bob will worry and take responsibility for safety, and Bob is rightly predicting that people like Alice are not worried.
This causes Alice and Bob...
Summary: I think it’s important for surveys about the future of technology or society to check how people's predictions of the future depend on their beliefs about what actions or responsibilities they and others will take on. Moreover, surveys should also help people to calibrate their beliefs about those responsibilities by collecting feedback from the participants about their individual plans. Successive surveys could help improve the groups calibration as people update their responsibilities upon hearing from each other. Further down, I’ll argue that not doing this — i.e. surveying only for predictions but not responsibilities — might even be actively harmful.
An example
Here's an example of the type of survey question combination I'm advocating for, in the case of a survey to AI researchers about the future impact of AI.
Prediction about impact:
1) Do you think AI development will have a net positive or net negative impact on society over the next 30 years?
Prediction about responsibility/action:
2) What fraction of AI researchers over the next 30 years will focus their full-time research attention on ensuring that AI is used for positive and rather than negative societal impacts?
Feedback on responsibility/action:
3) What is the chance that you, over the next 30 years, will transition to focusing your full-time research attention on ensuring that AI is used for positive rather than negative societal impacts?
I see a lot of surveys asking questions like (1), which is great, but not enough of (2) or (3). Asking (2) will help expose if people think AI will be good as a result of other people will take responsibility for making it good. Asking (3) will well help the survey respondents to update by seeing if their prediction in (2) matches the responses of other survey respondents in (3).
How this helps
I’ve seen it happen that everyone thinks something is fine because someone else will deal with it. This sort of survey could help folks to notice when that’s not the case. In other words, it could help mitigate the bystander effect.
Similarly, I’ve also seen it happen that everyone gets worried about a thing because they think no one else is doing anything about it, and then they go around taking a bunch of more-drastic-than-necessary unilateral actions. This sort of survey can help to mitigate this sort of social malcoordination. That is, it could help mitigate the “unilateralist’s curse” (which I think is essentially just the opposite of the bystander effect).
Finally, surveying to coordinate feels like a more cooperative and agentic game than just collecting socially contingent predictions about what will happen in the future, as though the future is inevitable. It ascribes agency, rather than merely predictive power, to the group of survey respondents as a whole. And it suggests what parameters are available for the group to change the future, namely, the allocation of certain responsibilities.
The side-taking effect: why surveying for predictions alone can be actively bad
More is true. I claim that without adding this sort of coordination information to the group’s discourse, surveys about prediction can sometimes sow seeds of deep-rooted disagreement that actually make coordination harder to achieve. Here’s how it works:
Alex: “Why worry about AI safety? It would be silly to make AI unsafe. Therefore someone will take responsibility for it.”
Bailey: “You should definitely worry about AI safety, because many people are not taking responsibility for it.”
These views are strangely compatible and therefore hard to reconcile by evidence alone. Specifically, Alice is rightly predicting that people like Bob will worry and take responsibility for safety, and Bob is rightly predicting that people like Alice are not worried.
This causes Alice and Bob to disagree with each other in a way that is fairly robust and difficult to settle without shifting the conversation to being about responsibilities instead of impact predictions. These persistent disagreements can result in factioning where people end up divided on whether they take the position that the responsibility in question (in the example, AI safety) is important or not. We end up with a lot of side-taking in favor of or against the responsibility, without a lot of discussion of how or when that responsibility will be distributed.
The organic version
It’s possible that surveys about prediction alone can still be net-good, because people naturally carry out discussions (2) and (3) slowly and organically on their own. For instance, I’ve given and seen others give talks about the neglectedness of AI safety as an area of research, by arguing from study results compiled by other researchers about the disparity between (a) the widespread opinion that AI safety is important, (b) the widespread opinion that AI safety will eventually we well-taken care of as a research area, and (b) the widespread lack of funding for the topic, at least prior to 2015.
But this sort of organic responsibility-awareness development can take years or decades; at least seems to be taking that that long in the case of “AI safety” as a responsibility. I’d like to see groups and communities develop a faster turnaround time for adopting and distributing responsibilities, and it seems to me like the sort of survey questions I’m proposing here can help with that.
My offer
If you’re a researcher who is already conducting a survey on the future of AI, even if you don't see a way to incorporate the sort of methodology I’m suggesting for the particular questions you’re asking, I'd love a chance to see the content you have planned, just in case I can come up with some suggestions myself. If you’re interested in that, you can email about your upcoming survey at critch+upcoming-surveys@eecs.berkeley.edu.
(Please don’t use this email for other topics besides surveys that are already definitely going to happen soon; I don’t have a lot of availability to create new initiatives right now.)
If your survey isn’t about AI but about some other impactful technological or societal change, I think I’m less likely to be able to add value to your thinking about it much beyond the writing of this post, but I might be willing to try anyway depending on my availability at the time.
Thanks for reading!