Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Jonathan_Graehl comments on The Best Textbooks on Every Subject - Less Wrong

167 Post author: lukeprog 16 January 2011 08:30AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (327)

You are viewing a single comment's thread.

Comment author: Jonathan_Graehl 17 January 2011 04:27:19AM *  7 points [-]

Subjects: algorithms/computational complexity, physics, Bayesian probability, programming

Introduction to Algorithms (Cormen, Rivest) is good enough that I read it completely in college. The exercises are nice (they're reasonably challenging and build up to useful little results I've recalled over my programming career). I think it's fine for self-study; I prefer it to the undergrad intro level or language-specific books. Obviously the interesting part about an algorithm is not the Java/Python/whatever language rendering of it. I also prefer it to Knuth's tomes (which I gave up on finishing - not enough fun). Knuth invents problems so he can solve them. He explains too much minutia. But his exercises are varied and difficult. If you like very hard puzzles, it's a good place to look.

Introduction to Automata Theory, Languages, and Computation (Hopcroft+Ullman) was also good enough for me to read. I've referred to it many times since. However, it's apparently not well-liked by others; maybe because it's too dense for them? I haven't read any other textbooks in the area.

The Feynman Lectures on Physics are also fun to read. But I doubt someone could use them as an intro course on their own. Because they're filled with entertaining tidbits, I was tempted to read through them without actually following the math 100%. Obviously this somewhat defeats the purpose. That's always a danger with well written technical material consumed for pleasure. I had already taken a few physics courses before I read Feynman; his lectures were better than the course textbooks (which I already forgot).

I didn't care for Jaynes. I only read about 700 pages, though. I remember there was some group reading effort that stopped showing up on LW after just a few chapters :)

For plain old programming, I've read quite a few books, and really liked The Practice of Programming - it was too short. I read Dijkstra's a discipline of programming and loved it for its idea to define program semantics precisely and to prove your code correct (nobody really practices this; it's too slow and hard compared to "debugging"), but it's probably not worth the price - I checked it out from a library.

I also agree with rwallace's recommendations also, except that the AI text is not especially useful (not that I know of a better one). I would not give SICP to a novice, though. Although I had done everything described in the book before (and already knew lisp), it did increase my appreciation of using closures and higher order functions as an alternative for the usual imperative/OO stuff. It also covers interpretation and compilation quite well (skipping the character-sequence parsing part - this is lisp, after all).

Comment author: lukeprog 17 January 2011 04:55:33AM 1 point [-]

Jonathan_Graehl,

Thanks for your recommendations, though I've set a rule that I won't add recommendations to the list in the original post unless those recommendations conform to the rules. Would you mind adding to what you've written above so as to conform to rule #3?

For example, you could list two other books on algorithms and explain why you prefer Introduction to Algorithms to those other books. And you could do the same for the subject of physics, and the subject of programming, and so on.

Comment author: gjm 18 January 2011 01:58:56AM 5 points [-]

Well, let me do Jonathan's job for him on one of those.

Introduction to Algorithms by Cormen, Leiserson, Rivest, and (as of the second edition) Stein is a first-rate single-volume algorithms text, covering a good selection of topics and providing nice clean pseudocode for most of what they do. The explanations are clear and concise. (Readers whose tolerance for mathematics is low may want to look elsewhere, though.)

Two obvious comparisons: Knuth's TAOCP is wonderful but: very, very long; now rather outdated in the range of algorithms it covers; describes algorithms with wordy descriptions, flowcharts, and assembly language for a computer of Knuth's own invention. When you need Knuth, you really need Knuth, but mostly you don't. Sedgwick's Algorithms (warning: it's many years since I read this, and recent editions may be different) is shallower, less clearly written, and frankly never gave me the same the-author-is-really-smart feeling that CLRS does.

(If you're going to get two algorithms books rather than one, a good complement to CLRS might be Skiena's "The algorithm design manual", more comments on which you can find on my website.)

Comment author: Jonathan_Graehl 26 January 2011 02:38:33AM 2 points [-]

Thanks. I really didn't have the ability to easily recall names of what few alternatives I've read (although in the area of programming in general, there are dozens of highly recommended books I've actually read - Design Patterns (ok), Pragmatic Programmer (ok), Code Complete (ok), Large Scale C++ Software Design (ok), Analysis Patterns (horrible), Software Engineering with Java (textbook, useless), Writing Solid Code (ok), object-oriented software construction (ok, sells the idea of design-by-contract), and I could continue listing 20 books, but what's the point. These are hardly textbooks anyway.

On algorithms, other than Knuth (after my disrecommendation of his work, I just bought his latest, "Combinatorial Algorithms, part 1"), really the only other one I read is "Data Structures in C" or some similar lower level textbook, which was unobjectionable but did not have the same quality.

Comment author: gjm 26 January 2011 04:09:41PM 0 points [-]

You're welcome! (Of the other books you mention that I've read, I agree with your assessment except that I'd want to subdivide the "ok" category a bit.)

Comment author: Mimi 19 March 2012 02:41:53AM *  0 points [-]

Manber's "Algorithms--a creative approach" is better than Cormen, which I agree is better than Knuth. It's also better than Aho's book on algorithms as well. It's better in that you can study it by yourself with more profit. On the other hand, Cormen's co-author has a series of video lectures at MIT's OCW site that you can follow along with.

Comment author: gjm 19 March 2012 08:18:36PM 2 points [-]

What about Manber's book makes it more fruitful for self-study than CLRS? How does it compare with CLRS in other respects? (Coverage of algorithms and data structures; useful pseudocode; mathematical rigour; ...)

Comment author: Davidmanheim 18 January 2011 03:38:52PM 0 points [-]

As a counterpoint to Hopcroft+Ullman, from another who has not read other books, Problem Solving in Automata, Languages, and Complexity by Ding-Zhu Du and Ker-I Ko was terrific. I did it as an undergraduate independent study class, completely from this book, and found it to be easy to follow if you are willing to work through problems.

Maybe we need someone who knows something more on the subject?

Comment author: Jonathan_Graehl 26 January 2011 02:47:37AM 0 points [-]

Hopcroft+Ullman is very proof oriented. Sometimes the proof is constructive (by giving an algorithm and proving its correctness). I liked it. There may be much better available for self-study.

Specialty algorithms: I briefly referenced Numerical Optimization and it seems better than Numerical Recipes in C. I didn't read it cover to cover.

Algorithms on Strings, Trees, and Sequences (Gusfield) was definitely a good source for computational biology algorithms (I don't do computation biology, but it explains fairly well things like suffix trees and their applications, and algorithms matching a set of patterns against substrings of running text).

Foundations of Natural Language Processing is solid. I don't think there's a better textbook (for the types of dumb, statistics/machine-learning based, analysis of human speech/text that are widely practiced). It's better than "Natural Language Understanding" (Allen), which is more old-school-AI.

Comment author: Mimi 19 March 2012 03:14:29AM 1 point [-]

For an AI text, I think any (text)book on a subject of your interest by Judea Pearl would fit the bill.
"Symbolic Logic and Mechanical Theorem Proving" by Chang and Lee is still an exceptionally lucid introduction to non-probabilistic AI.

Comment author: Mimi 19 March 2012 03:01:27AM 1 point [-]

I also prefer Hopcroft+Ullman (original edition) to later alternatives like their own later edition, Papadimitriou, and even Sipser who is widely regarded as having written the definitive intro text.

Comment author: Mimi 19 March 2012 02:56:29AM 1 point [-]

"A Discipline of Programming" is rather hard to follow. Dromey gives an introductory treatment that's a bit too introductory, "Progamming Pearls" by Bently includes another even more informal treatment, and Gries's "Science of Programming" would be the textbook version that I might recommend covering this material. All three are somewhat dated. More modern treatment would be either Apt's "Verification of Sequential and Concurrent Programs" or Manna's "The Calculus of Computation." and depending on your focus one would be better than the other. However, the ultimate book I would recommend in this field is "Interactive Theorem Proving and Program development" by Yves Bertot. It doesn't teach Hoare's invariant method like the other books, but uses a more powerful technique in functional programming for creating provably correct software.

Comment author: Jonathan_Graehl 19 March 2012 09:42:53PM 0 points [-]

I'll look for Bertot's book. I agree that "A Discipline" is not a pleasant read (though I found it rewarding).