= 27d567bc2d48d9f40e9ccc20ea370b15)
gwern comments on "Human-level control through deep reinforcement learning" - computer learns 49 different games - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
Comments (19)
I saw this paper before, and maybe I'm being an idiot but I didn't understand this:
I thought one generally trained the networks layer by layer, so layer n would be completely finished training before layer n+1 starts. Then there is no problem of "the distribution of each layer's inputs changes" because the inputs are fixed once training starts.
Admittedly, this is a problem if you don't have all the training data to start of with and want to learn incrementally, but AFAICT that is not generally the case in these benchmarking contests.
Regardless, its amazing how simple DNNs are. People have been working on computer vision and AI for about 60 years, and then a program like this comes along which is only around 500 lines of code, conceptually simple enough to explain to anyone with a reasonable mathematical background, but can nevertheless beat humans at a reasonable range of tasks.
I get the impression it's a hardware issue. See for example http://nautil.us/issue/21/information/the-man-who-tried-to-redeem-the-world-with-logic - McCulloch & Pitts invented neural networks almost before digital computers existed* and he was working on "three-dimensional neural networks". They didn't invent backpropagation, I don't think, but even if they had, how would they have run, much less trained, the state of the art many-layer neural networks with millions of nodes and billions of connections like we're seeing these days? What those 60 years of work gets you is a lot of specialized algorithms which don't reach human-parity but at least are computable on the hardware of that day.
* depends on what exactly you consider the first digital computer and how long before the key publication you date their breakthrough.
Can confirm that hardware (and data!) are the two main culprits here. The actual learning algorithms haven't changed much since the mid 1980s, but computers have gotten many times faster, GPUs are 30-100x faster still, and the amount of data has similarly increased by several orders of magnitude.