It's difficult to get the latter without the former, if you want to make successful way-out-of-sample predictions.

I think you can do a lot of successful predictions with IQ without knowing the mechanism of IQ. I don't think you build better IQ tests by going into neuroscience but giving the tests to people and seeing how different variables correlate with each other.

Otherwise, you're stuck in the morass of trying to find tiny signals and dismissing most of your data as noise.

I don't think that's true. The present approach of putting compounds through massive screening arrays based on theoretical reasoning that it's good to hit certain biochemical pathways is very noise-laden and produces a lot of false positives. >90% of drug candidats that get put into trials don't work out.