Open thread, Nov. 16 - Nov. 22, 2015

MrMind

Open thread, Nov. 16 - Nov. 22, 2015 — LessWrong

Comment Permalink

I've been hearing about all this amazing stuff done with recurrent neural networks, convolutional neural networks, random forests, etc. The problem is that it feels like voodoo to me. "I've trained my program to generate convincing looking C code! It gets the indentation right, but the variable use is a bit off. Isn't that cool?" I'm not sure, it sounds like you don't understand what your program is doing. That's pretty much why I'm not studying machine learning right now. What do you think?

Showing 3 of 12 replies (Click to show all)

V_V11y00

The trippy pictures and the vaguely C-looking code are just cool stunts, not serious experiments. People may be tempted to fell into the hype, sometimes a reality check is helpful.

This said, neural networks really do well in difficult tasks such as visual object recognition and machine translation, indeed for reasons that are not fully understood.

Sounds like a good reason to study the field in order to understand why they can do what they do, and why they can't do what they can't do, doesn't it?

2bogus11y

What this is really saying is: "Hey, convincing-looking C code can be modeled by a RNN, i.e. a state-transition version ("recurrent") of a complex non-linear model which is ultimately a generalization of logistic regression ("neural network")! And the model can be practically 'learned', i.e. fitted empirically, albeit with no optimality or accuracy guarantees of any kind. The variable use is a bit off, though. Isn't this cool/Does this tell us anything important?"

10passive_fist11y

I did my PhD thesis on a machine learning problem. I initially used deep learning but after a while I became frustrated with how opaque it was so I switched to using a graphical model where I had explicitly defined the variables and their statistical relationships. My new model worked but it required several months of trying out different models and tweaking parameters, not to mention a whole lot of programming things from scratch. Deep learning is opaque but it has the advantage that you can get good results rapidly without thinking a lot about the problem. That's probably the main reason that it's used.

See in context