You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

jacob_cannell comments on Open Thread, Jun. 29 - Jul. 5, 2015 - Less Wrong Discussion

5 Post author: Gondolinian 29 June 2015 12:14AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (210)

You are viewing a single comment's thread. Show more comments above.

Comment author: jacob_cannell 02 July 2015 12:28:10AM 2 points [-]

There is the probabilistic programming community which uses clean tools (programming languages) to hand construct models with many unknown parameters. They use approximate bayesian methods for inference, and they are slowly improving the efficiency/scalability of those techniques.

Then there is the neural net & optimization community which uses general automated models. It is more 'frequentist' (or perhaps just ad-hoc ), but there are also now some bayesian inroads there. That community has the most efficient/scalable learning methods, but it isn't always clear what tradeoffs they are making.

And even in the ANN world, you sometimes see bayesian statistics brought in to justify regularizers or to derive stuff - such as in variational methods. But then for actual learning they take gradients and use SGD, with the understanding that SGD is somehow approximating the bayesian inference step, or at least doing something close enough.