You assume that economists are actually an expert on the economy. They aren't. That's the problem.

Economics only really has a good understanding of very low level effects, and even there things are very difficult to truly deal with. The law of supply and demand, for instance, is really less of a law and more of a guideline - the only way to actually determine real world behavior is experimentation, as there is no single equation you can plug things into to get a result out of. And that's something SIMPLE. Ask them how to fix the economy? They have no ability to do that because they don't actually understand the economy, nor how it works.

A great deal of economics is down to ideology rather than actual reality. There are a lot of economic models but they either only represent very narrow areas or they don't actually match up with reality. Ask ten economists what happens when you raise minimum wage and you'll get ten answers - three will agree one way, three will agree the other, and the other four will give you two answers each.

The problem is that the economy is such a hideously complex system that economists have no way to actually simulate it or experiment with it (unless you argue what is actually done is an experiment, but if it is, it is a very poorly controlled one with a ridiculous number of variables).

Indeed, if you look at reality, it is obvious that economists are NOT experts on the economy. How can you tell this? Because economists make their money by being paid to be economists.

The best economist in the world is probably Warren Buffet, because he makes his money by actually making predictions about the economy and then makes money based on how good they are (as well as how well he manages his business). But ask him about tax policy and he'll still likely not have any better answer than you can get via thought and intuition.

Empirical evidence suggests that economists are not experts on the economy.

One friend told me that economics as a field has a huge "conservative bias", and you can't trust their methods because of this reason.

Which is strange, since (according to The Making of an Economist, Redux by David Colander) only 16% of economists are conservative (while 47% are liberal, 24% are moderate, and 6% are radical). Perhaps he meant that the field of economics has less of a non-conservative bias than other fields (like sociology and political science), which I'm pretty sure is true.

For yourself? Study economics until you can properly trust or distrust the experts.

For other people, ignore or study, but don't engage.

I notice that people (including me) are especially prone to this in economics.

First, no area is as bad as sports :) Now, I don't know much about economics, but let us assume for the moment that it is a real natural science and it is impossible to get to the leading edge without investing thousands of hours of hard work, then repeating and alieving the mantra I have suggested in the OP would be my recommendation. One way to convert this belief into alief could be looking through the relevant textbooks and realizing that you cannot possibly understand the advanced stuff without learning the basics first. And real-life economic policy is probably as advanced as it gets. Sort of an economic version of this.

Now, it is not necessarily true that economics is like other natural sciences, or like law or medicine. Maybe it is more like alchemy and what the subject matter experts learn over a decade or two does not help at all with giving sound economic advice. Maybe one or two terms is enough, and the rest is just fluff. I don't know. But this point ought to be addressed first, somehow...