Human performance on image-recognition surpassed by MSR? "Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification", He et al 2015 (Reddit; emphasis added):
Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier neural networks for image classification from two aspects. First, we propose a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit. PReLU improves model fitting with nearly zero extra computational cost and little overfitting risk. Second, we derive a robust initialization method that particularly considers the rectifier nonlinearities. This method enables us to train extremely deep rectified models directly from scratch and to investigate deeper or wider network architectures. Based on our PReLU networks (PReLU-nets), we achieve 4.94% top-5 test error on the ImageNet 2012 classification dataset. This is a 26% relative improvement over the ILSVRC 2014 winner (GoogLeNet, 6.66%). To our knowledge, our result is the first to surpass human-level performance (5.1%, Russakovsky et al.) on this visual recognition challenge.
(Surprised it wasn't a Baidu team who won.) I suppose now we'll need even harder problem sets for deep learning... Maybe video? Doesn't seem like a lot of work on that yet compared to static image recognition.
The record has apparently been broken again: "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift" (HN, Reddit), Ioffe & Szegedy 2015:
...Training Deep Neural Networks is complicated by the fact that the distribution of each layer’s inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to thi
If it's worth saying, but not worth its own post (even in Discussion), then it goes here.
Previous Open Thread
Next Open Thread
Notes for future OT posters:
1. Please add the 'open_thread' tag.
2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)
3. Open Threads should be posted in Discussion, and not Main.
4. Open Threads should start on Monday, and end on Sunday.