(nods) Yup. Of course, McArdle's claims about what people would have said before the study, if asked, are also only being made after the results are known, which as you say is a classic failure mode.

Of course, McArdle is neither passing laws nor doing research, just writing articles, so the cost of failure is low. And it's kind of nice to see someone in the mainstream (sorta) press making the point that surprising observations should change our confidence in our beliefs, which people surprisingly often overlook.

Anyway, the quality of McArdle's analysis notwithstanding, one place this sort of reasoning seems to lead us is to the idea that when passing a law, we ought to say something about what we anticipate the results of passing that law to be, and have a convention of repealing laws that don't actually accomplish the thing that we said we were passing the law in order to accomplish.

Which in principle I would be all in favor of, except for the obvious failure mode that if I personally don't want us to accomplish that, I am now given an incentive to manipulate the system in other ways to lower whatever metrics we said we were going to measure. (Note: I am not claiming here that any such thing happened in the Oregon study.)

That said, even taking that failure mode into account, it might still be preferable to passing laws with unarticulated expected benefits and keeping them on the books despite those benefits never materializing.