I notice that people (including me) are especially prone to this in economics.
First, no area is as bad as sports :) Now, I don't know much about economics, but let us assume for the moment that it is a real natural science and it is impossible to get to the leading edge without investing thousands of hours of hard work, then repeating and alieving the mantra I have suggested in the OP would be my recommendation. One way to convert this belief into alief could be looking through the relevant textbooks and realizing that you cannot possibly understand the advanced stuff without learning the basics first. And real-life economic policy is probably as advanced as it gets. Sort of an economic version of this.
Now, it is not necessarily true that economics is like other natural sciences, or like law or medicine. Maybe it is more like alchemy and what the subject matter experts learn over a decade or two does not help at all with giving sound economic advice. Maybe one or two terms is enough, and the rest is just fluff. I don't know. But this point ought to be addressed first, somehow...
Now, it is not necessarily true that economics is like other natural sciences, or like law or medicine.
I'm confused by this categorization of law.
I would suggest that economics is not best categorized as a science, since models are rarely tested and then discarded on falsification (real life conditions are rarely good enough at isolating variables to convince proponents that a hypothesis has seriously been falsified,) but that good economists probably do have expertise that an interested amateur couldn't duplicate with a few key insights. On the other ...
So, one more litany, hopefully someone else finds it as useful.
It's an understatement that humility is not a common virtue in online discussions, even, or especially when it's most needed.
I'll start with my own recent example. I thought up a clear and obvious objection to one of the assertions in Eliezer's critique of the FAI effort compared with the Pascal's Wager and started writing a witty reply. ...And then I stopped. In large part because I had just gone through the same situation, but on the other side, dealing with some of the comments to my post about time-turners and General Relativity by those who know next to nothing about General Relativity. It was irritating, yet here I was, falling into the same trap. And not for the first time, far from it. The following is the resulting thought process, distilled to one paragraph.
I have not spent 10,000+ hours thinking about this topic in a professional, all-out, do-the-impossible way. I probably have not spent even one hour seriously thinking about it. I probably do not have the prerequisites required to do so. I probably don't even know what prerequisites are required to think about this topic productively. In short, there are almost guaranteed to exist unknown unknowns which are bound to trip up a novice like me. The odds that I find a clever argument contradicting someone who works on this topic for a living, just by reading one or two popular explanations of it are minuscule. So if I think up such an argument, the odds of it being both new and correct are heavily stacked against me. It is true that they are non-zero, and there are popular examples of non-experts finding flaws in an established theory where there is a consensus among the experts. Some of them might even be true stories. No, Einstein was not one of these non-experts, and even if he were, I am not Einstein.
And so on. So I came up with the following, rather unpolished mantra:
If I think up what seems like an obvious objection, I will resist assuming that I have found a Weaksauce Weakness in the experts' logic. Instead I may ask politely whether my argument is a valid one, and if not, where the flaw lies.
If you think it useful, feel free to improve the wording.