Besides what fezziwig said, which is correct, the other issue is the fundamental capabilities of the domain you are looking at. I figured something like this was the source of the error, which is why I asked for context. Neural networks, deep or otherwise, are basically just classifiers. The reason we've seen large advancements recently in machine learning is chiefly because of the immense volumes of data available to these classifier-learning programs. Machine learning is particularly good at taking heaps of structured or unstructured data and finding clusters, then coming up with ways to classify new data into one of those identified clusters. The more data you have, the most detail that can be identified, and the better your classifiers become. Certainly you need a lot of hardware to process the mind boggling amounts of data that are being pushed through these machine learning tools, but hardware is not the limiter, available data is. Giant companies like Google and Facebook are building better and better classifiers not because they have more hardware available, but because they have more data available (chiefly because we are choosing to escrow our personal lives to these companies servers, but that's an aside). In as much as machine learning tends to dominate current approaches to narrow AI, you could be excused for saying "the biggest limitation on AI development is availabilities of data." But you mentioned safety, and AI safety around here is a codeword for general AI, and general AI is truly a software problem that has very little to do with neural networks, data availability, or hardware speeds. "But human brains are networks of neurons!" you reply. True. But the field of computer algorithms called neural networks is a total misnomer. A "neural network" is an algorithm inspired by an over simplification of a misconception of how brains worked that dates back to the 1950's / 1960's. Developing algorithms that are actually capable of performing general i

I think you have the right idea, but it's a mistake to conflate "needs a big corpus of data" and "needs lots of hardware". Hardware helps, the faster the training goes the more experiments you can do, but a lot of the time the gating factor is the corpus itself. For example, if you're trying to train a neural net to solve the "does this photo contain a bird?" problem, you need a bunch of photos which vary at random on the bird/not-bird axis, and you need human raters to go through and tag each photo as bird/not-bird. There are many ways to lose here. For example, your variable of interest might be correlated to something boring (maybe all the bird photos were taken in the morning, and all the not-bird photos were taken in the afternoon), or your raters have to spend a lot of time with each photo (imagine you want to do beak detection, instead of just bird/not-bird: then your raters have to attach a bunch of metadata to each training image, describing the beak position in each bird photo).