That is too sweeping a criticism. We have to model the world somehow when making decisions, and quantitatively. That goes for policy makers also.
With all due respect, this is exactly the sort of stonewalling, discourse-killing response I commonly see from economists. If you can't justify your model, then the correct and honest thing to do is to admit that you don't have a model and try to deal as best you can with your ignorance -- not to continue using models divorced from reality with the excuse that you don't have anything better. An astrologer can use the same excuse, with equal justification.
However, clearly, admitting ignorance in these matters also means admitting that "scientific" policy is in many instances just pseudoscientific dilettantism, which in turn hits all sorts of ideological hot buttons. Of course, some centuries ago, a court astrologer would also likely stonewall if questioned too much about the validity of his astrology, which is an almost perfect analogy.
Are you familiar with Jared Diamond's lay account and defense of operationalization, here? I'd be interested to hear what parts of it you object to.
There is in principle no problem with operationalization, as long as the models based on it are sound logically and epistemologically. In particular, it is important that the operationalized concepts can be traced back to how exactly they correspond to observable reality, and that the limitations of models are understood and recognized by those who employ them. What I'm arguing is that many concepts used by economists don't satisfy these basic criteria of validity.
The crucial practical problem here is that attempts to tackle scientifically issues that are relevant for politics, power, and ideology are more likely to lead to the emergence of ideologically-driven pseudoscience than to a real scientific clarification of contentions issues. Clearly, modern economics and other social sciences have not been immune to this problem, which is probably the main reason for their present awful state.
There are two questions here. One is whether economists have a good influence on policy. Another is whether they even understand anything. I don't have any illusions about the first question, but my interest is in understanding.
We observe that prices tend to go up, and wish to quantify that observation, maybe to test ideas about its relationship to money supply. So we have to operationalize "prices go up." We do it by sampling commonly purchased things and keeping track of their prices over time. That seems to me to be as innocuous as opera...
A Wall Street Journal article by Harvard professor of government Harvey Mansfield claims that the social sciences and humanities are inferior to the sciences. The article implicitly urges undergraduates to major in science. From the article:
Do you agree with this? As a game theorist I probably have a rather biased view of the situation. It's certainly true that the ideal of the scientific method is vastly better than the practice of economists, but I think that majoring in economics provides better training for a rationalist than majoring in any of the sciences does.
Economics explicitly considers what it means to be rational. Although it infrequently considers ways in which humans are irrational, I'm under the impression that the hard sciences never do this. Furthermore, because economists can almost never perform replicable experiments we have to rely on what everyone in the profession recognizes as messy data; therefore we’re far more equipped than hard scientists to understand the limits of using statistical inference to draw conclusions from real world situations. Although I have seen no data on this, I bet that a claim by nutritionists that they have found a strong causal link between some X and heart disease would be treated with far more skepticism by the average economist than the average hard scientist.