Factor Analysis is a very flawed technique for many circumstances; in particular, "independent" and "not linearly correlated" are wildly different.

So yes, try to be acutely aware that not everything is a nail. Be aware of your hammer's limitations, and what it's effectiveness is based on.

That said, I've been guilty of this too. And certainly, we rely on people pushing their hammers in strange directions; some esoteric new hammering technique for metaphorical screws can open the floodgates of innovation. That said, don't confuse the fact that some people have managed to hone their skills with one tool with that tool being a panacea.