Part of the problem is that changes in X might be aimed at keeping Y stable when some other factor Z varies. See Milton Friedman's Thermostat:

If the driver is doing his job right, and correctly adjusting the gas pedal to the hills, you should find zero correlation between gas pedal and speed, and zero correlation between hills and speed.

For example, it's hard to answer the question "do armies stop invasions?" by using correlations, because the rulers can adjust the strength of the army in response to the risk of getting invaded, so the resulting risk depends mostly on the risk tolerance of the rulers.