magfrump

A neural net using rectified linear unit activation functions of any size is unable to approximate the function sin(x) outside a compact interval. I am reasonably confident that I can prove that any NN with ReLU activation approximates a piecewise linear function. I believe the number of linear pieces that...

This post is a response to the recent Astral Codex Ten post, “CHAI, Assistance Games, And Fully-Updated Deference”. A brief summary of the context, for any readers who are not subscribed to ACX or familiar with the shutdown problem: The Center for Human-Compatible Artificial Intelligence (CHAI) is a research group...

cross-posted from my blog Background on the events I have been thinking about this since the firing of Dr. Timnit Gebru, and yet still no one has actually written about it beyond my own tweets, so I guess it falls to me. I find, and I imagine many people in...

cross-posted from my personal blog I have an intuition that the gap between short term ML alignment problems and long term AGI alignment problems is smaller than usual. Writing on the subject I usually see rarely takes this perspective. I see a few things actually get written: * People who...

One of the dominant paradigms in image processing today is transfer learning from well established image classification algorithms (such as DenseNet, VGG, or Inception) to other image processing applications. The idea is that in the process of learning to classify types of images, the networks learn to recognize distinctive visual...

Maybe deep learning researchers are just like mathematicians. They think of coding as the hard part of what they do so they spend a lot of time teaching people how to implement neural nets from scratch. But actually the hard part of what they do is math (or at least...

When I was an undergraduate considering going to graduate school in mathematics, one thing that I knew a reasonably amount about and spent a good amount of time thinking about was what the field of math looked like overall. In very, very broad strokes I knew that geometry, algebra, and...