More Is Different for AI

Machine learning is touching increasingly many aspects of our society, and its effect will only continue to grow. Given this, I and many others care about risks from future ML systems and how to mitigate them.

I've distilled my thoughts into a series of blog posts, where I'll argue that:

1. Future ML Systems Will be Qualitatively Different from those we see today. Indeed, ML systems have historically exhibited qualitative changes as a result of increasing their scale. This is an instance of "More Is Different", which is commonplace in other fields such as physics, biology, and economics (see Appendix: More Is Different in Other Domains). Consequently, we should expect ML to exhibit more qualitative changes as it scales up in the future.
2. Most discussions of ML failures are anchored either on existing systems or on humans. Thought Experiments Provide a Third Anchor, and having three anchors is much better than having two, but each has its own weaknesses.
3. If we take thought experiments seriously, we end up predicting that ML Systems Will Have Weird Failure Modes. Some important failure modes of ML systems will not be present in any existing systems, and might manifest quickly enough that we can't safely wait for them to occur before addressing them.
4. My biggest disagreement with the Philosophy view is that I think Empirical Findings Generalize Surprisingly Far, meaning that well-chosen experiments on current systems can tell us a lot about future systems.