Important policies have so many effects that it is near impossible to keep track of them all. In addition, some effects tend to dwarf all others, so it is critical to catch every last one. (Perhaps they follow a Paretian distribution?) It follows that any quantitative analysis of policy effects tends to be seriously flawed.
I don't think this is the right way to frame the problem.
It is true that even unimportant policies have so many effects that it is de-facto impossible to calculate them all. And it is true that one or a few effects tend to dwarf all others. But that does not mean that it's critical to catch every last one. The effects which dwarf all others will typically be easier to notice, in some sense, precisely because they are big, dramatic, important effects. But "big/important effect" is not necessarily the same as "salient effect", so in order for this work in practice, we have to go in looking for the big/important effects with open eyes rather than just asking the already-salient questions.
For instance, in the pot/IQ example, we can come at the problem from either "end":
If people think about the problem in a principled way like this, then I expect they'll come up with hypotheses like the pot-IQ thing. There just aren't that many things which are highly important to humans in the aggregate, or that many things on which any given variable has a large expected effect. (Note the use of "expected effect" - variables may have lots of large effects via the butterfly effect, but that's relevant to decision-making only insofar as we can predict the effects.)
The trick is that we have to think about the problem in a principled way from the start, not just get caught up in whatever questions other people have already brought to our attention.
johnswentworth makes the great point that "some effects tend to dwarf all others, so it is critical to catch every last one" assumes that we can't identify the big effects early. If people are looking around with open eyes, they're not so unable to pick up the relevant stuff first.
What yhoiseth's framing gets right is that big effects are sometimes not salient, even for people with open eyes. And especially when effects are hard to directly observe or estimate with certainty because they're indirect in nature (like substitution effects), not only are...
It seems similar to what Andrew Gelman has called the piranha problem (two links there). Also related is Gelman's kangaroo.
We could dub this "Long Tail Externalities" - the idea that most of the impact comes from a few indirect effects, and sometimes the more indirect the bigger - for instance, most policies might impact the future mainly through AI safety.
In Things I learned Writing The Lockdown Post, Scott Alexander describes a really tricky issue when trying to quantify the effects of some policies:
There is more, but this covers the phenomenon I’m curious about. Let me try to describe the problem in general terms:
Important policies have so many effects that it is near impossible to keep track of them all. In addition, some effects tend to dwarf all others, so it is critical to catch every last one. (Perhaps they follow a Paretian distribution?) It follows that any quantitative analysis of policy effects tends to be seriously flawed.
Do we already have a term for this problem? It reminds me of moral cluelessness as well as known and unknown unknowns, but none of those seem fit the bill exactly.