AutoBound on neural network can achieve OOMs lower training loss
Seems like gradient descent methods weren't using the relevant math bounds so far. Google released AutoBound as an open-source library. Here is what I consider a money shot of the article (notice it's a log-plot): Performance of SafeRate when used to train a single-hidden-layer neural network on a subset of...
Apr 17, 202310