Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

jsalvatier comments on The Best Textbooks on Every Subject - Less Wrong

167 Post author: lukeprog 16 January 2011 08:30AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (327)

You are viewing a single comment's thread.

Comment author: jsalvatier 17 January 2011 04:01:51PM *  17 points [-]

Update see my comment for new thoughts

Topic: Introductory Bayesian Statistics (as distinct from more advanced Bayesian statistics)

Recommendation: Data Analysis: A Bayesian Tutorial by Skilling and Sivia

Why: Sivia's book is well suited for smart people who have not had little or no statistical training. It starts from the basics and covers a lot of important ground. I think it takes the right approach, first doing some simple examples where analytical solutions are available or it is feasible to integrate naively and numerically. Then it teaches into maximum likelihood estimation (MLE), how to do it and why it makes sense from a Bayesian perspective. I think MLE is a very very useful technique, especially so for engineers. I would overall recommend just Part I: The Essentials, I don't think the second half is so useful, except perhaps the MLE extensions chapter. There are better places to learn about MCMC approximation.

Why not other books?

Bayesian Data Analysis by Gelman - Geared more for people who have done statistics before.

Bayesian Statistics by Bolstad - Doesn't cover as much as Sivia's book, most notably doesn't cover MLE. Goes kinda slowly and spends a lot of time on comparing Bayesian statistics to Frequentist statistics.

The Bayesian Choice - more of a mathematical statistics book, not suited for beginners.

Comment author: jsalvatier 20 September 2011 04:13:32PM *  8 points [-]

Brandon Reinhart used both Sivia's book and Bolstad's book and found (3rd message) Bolstad's book better for those with no stats experience:

For statistics, I recommend An Introduction to Bayesian Statistics by William Bolstad. This is superior to the "Data Analysis" book if you're learning stats from scratch. Both "Data Analysis" and "Bayesian Data Analysis" assume a certain base level of familiarity with the material. The Bolstad book will bootstrap you from almost no familiarity with stats through fairly clear explanations and good supporting exercises.

Nonetheless, it's something you should do with other people. You may not notice what you aren't completely comprehending otherwise. Do the exercises!

Based on these comments, I think I was underestimating inferential distance, and I now change my recommendation. You should read Bolstad's book first (skipping the parts comparing bayesian and frequentist methods unless that's important to you) and then read Sivia's book. If you have experience with statistics you may start with Sivia's book.

Comment author: lukeprog 17 January 2011 07:34:35PM 1 point [-]


Comment author: XFrequentist 12 August 2013 07:36:29PM 0 points [-]

There are better places to learn about MCMC approximation.

Any in particular? I came to this thread seeking exactly this.

Comment author: jsalvatier 13 August 2013 07:25:13PM 1 point [-]

I don't have an especially awesome place, but Bayesian Data Analysis by Gelman introduces the basics of Metropolis Hastings and Gibbs Sampling (those are probably the first ones to learn). There are probably quite a few other places to learn about these two algorithms too (including wikipedia). MCMC using Hamiltonian Dynamics by Neal, is the standard reference for Hamiltonian Monte Carlo (what I would suggest learning after those two).

Comment author: beoShaffer 05 November 2011 04:29:46AM 0 points [-]

Is Gleman's book a good recommendation for people who have done frequentist statistics and/or combinatorics? I have free access to it and basic familiarity with both.