I notice that people (including me) are especially prone to this in economics. Everyone feels that they can figure out the correct tax/foreign/monetary/regulatory policy simply by using some trivial demand-supply arguments. Further, they always use a moral axis as well (arguing about fairness/equality/social justice), which always dilutes whatever little insight they might have had.
Worse: economists are highly distrusted as experts. One friend told me that economics as a field has a huge "conservative bias", and you can't trust their methods because of this reason. Chalk it up to politics being the mind-killer.
Any suggestions on how to handle these situations?
You assume that economists are actually an expert on the economy. They aren't. That's the problem.
Economics only really has a good understanding of very low level effects, and even there things are very difficult to truly deal with. The law of supply and demand, for instance, is really less of a law and more of a guideline - the only way to actually determine real world behavior is experimentation, as there is no single equation you can plug things into to get a result out of. And that's something SIMPLE. Ask them how to fix the economy? They have no abili...
So, one more litany, hopefully someone else finds it as useful.
It's an understatement that humility is not a common virtue in online discussions, even, or especially when it's most needed.
I'll start with my own recent example. I thought up a clear and obvious objection to one of the assertions in Eliezer's critique of the FAI effort compared with the Pascal's Wager and started writing a witty reply. ...And then I stopped. In large part because I had just gone through the same situation, but on the other side, dealing with some of the comments to my post about time-turners and General Relativity by those who know next to nothing about General Relativity. It was irritating, yet here I was, falling into the same trap. And not for the first time, far from it. The following is the resulting thought process, distilled to one paragraph.
I have not spent 10,000+ hours thinking about this topic in a professional, all-out, do-the-impossible way. I probably have not spent even one hour seriously thinking about it. I probably do not have the prerequisites required to do so. I probably don't even know what prerequisites are required to think about this topic productively. In short, there are almost guaranteed to exist unknown unknowns which are bound to trip up a novice like me. The odds that I find a clever argument contradicting someone who works on this topic for a living, just by reading one or two popular explanations of it are minuscule. So if I think up such an argument, the odds of it being both new and correct are heavily stacked against me. It is true that they are non-zero, and there are popular examples of non-experts finding flaws in an established theory where there is a consensus among the experts. Some of them might even be true stories. No, Einstein was not one of these non-experts, and even if he were, I am not Einstein.
And so on. So I came up with the following, rather unpolished mantra:
If I think up what seems like an obvious objection, I will resist assuming that I have found a Weaksauce Weakness in the experts' logic. Instead I may ask politely whether my argument is a valid one, and if not, where the flaw lies.
If you think it useful, feel free to improve the wording.