"What I Learning Losing a Million Dollars" by Jim Paul and Brendan Moynihan (1994)

Subject: Analysis of catastrophic trading mistakes woven through the autobiography of a highly confident commodities trader. He made $250k in one day. Thereafter he went on to take greater risks in commodities markets, counting his profits before they were realized. At one point he considered renting a Concord jet to celebrate his imagined gains. Over the course of several months, however, due to a combination of misfortune, hubris , denial, and creative rationalization, his entire position was wiped out and he lost over $1 million.

This is the only non-quack popular book of its kind I know of which deconstructs the psychological mistakes a trader can make. The book is self-published, and until recently, was out of print. Sadly, one of the authors, Mr. Paul, perished in the attacks on the WTC. Not widely known about, the book is fortunately avalable once again.

The lessons contained in it are related not just to trading, but to life in general. Honest and unflinching in its analysis of self-deception, overconfidence, and how to guard against them. This book is unique.

The Black Swan by Nassim Nicholas Taleb.

Subject: The unexpected, and how people try and frequently fail spectacularly to predict and deal with it.

Example quote (from 2007): Globalization creates interlocking fragility, while reducing volatility and giving the appearance of stability. In other words it creates devastating Black Swans. We have never lived before under the threat of a global collapse. Financial Institutions have been merging into a smaller number of very large banks. Almost all banks are interrelated. So the financial ecology is swelling into gigantic, incestuous, bureaucratic banks – when one fails, they all fall. The increased concentration among banks seems to have the effect of making financial crisis less likely, but when they happen they are more global in scale and hit us very hard. We have moved from a diversified ecology of small banks, with varied lending policies, to a more homogeneous framework of firms that all resemble one another. True, we now have fewer failures, but when they occur ....I shiver at the thought.

Flawed, but well worth reading.

The Logic of Failure, by Dietrich Doerner.

Summary: Doerner draws on historical and laboratory data to illustrate generic features of many/most (I'd say "all") failures of judgment in complex situations. He offers suggestions on how to overcome our predisposition to failure in these situations.

Lessons: This book is a treasure trove. It ranges broadly -- complexity, goals, models, time, ignorance, planning, and more. I can't emphasize enough how illuminating (and readable!) this book is.

Here are some quotes from my notes on the first half of the book: "When we fail to solve a problem, we fail because we tend to make a small mistake here, a small mistake there, and these mistakes add up." (p. 7), "...it is far from clear whether "good intentions plus stupidity" versus "evil intentions plus intelligence" have wrought more harm in the world." (p. 8), "If, the moment something goes wrong, we no longer hold ourselves responsible but push the blame onto others, we guarantee that we remain ignorant of the real reasons for poor decisions, namely inadequate plans and failure to anticipate the consequences." (p. 27), "We find an inability to think in terms of nonlinear networks of causation rather than chains of causation -- an inability, that is, to properly assess the side effects and repercussions of one's behavior. We find an inadequate understanding of exponential development, an inability to see that a process that develops exponentially will, once it has begun, race to its conclusion with incredible speed. These are all mistakes of cognition." (p. 33), "[Characteristics of analysis of complicated systems:] complexity, intransparence, internal dynamics" (p. 37), "...the ability to make allowances for incomplete and incorrect information and hypotheses is an important requirement for dealing with complex situations." (p. 42), "Formless collections of data about random aspects of a situation merely add to the situation's impenetrability and are no aid to decision making." (p. 44), "...goals may be: positive or negative, general or specific, clear or unclear, simple or multiple, implicit or explicit" (p. 52), "By labeling a bundle of problems with a single conceptual label, we make dealing with that problem easier -- provided we're not interested in solving it. Phrases like "urgently needed measures for combating unemployment" roll easily off the tongue if we don't have to do anything about unemployment." (p. 55), "There are many ways of tackling multiple problems at once, but the one thing we usually cannot do is solve all the problems at once." (p. 55), "Goals conflict with one another not by their very nature but because the variables relating to them in the system are negatively linked." (p. 57), "[The problems of DDT (e.g.)] So the mistake is less not knowing than not wanting to know. And not wanting to know is a result not of ill will or egoism but of thinking that focuses on an immediately acute problem." (p. 58)

Psychology of Intelligence Analysis, by Richards Heuer

Summary: Intelligence analysts cope with the need to aggregate heterogeneous data of wildly varying quality into coherent decision-making frameworks. This book catalogues historical failures made in this sort of analytical context and suggests how to overcome common traps.

Lessons: This is a great guide from a sharp practitioner. Analysts wanting to draw conclusions from data should generate a wide range of hypotheses and use falsification to narrow down to a smaller range of options. Such an approach avoids the trap of establishing a single theory early in the analytical process and only paying attention to evidence that confirms it.

Note: I spoke with the author a few months ago, and he said a dramatically revised edition was in the works. I'm drawing heavily on this book for two chapters of my forthcoming book, Head First Data Analysis (O'Reilly Media).

Here's a free PDF of the whole thing: https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/psychology-of-intelligence-analysis/index.html

Since we are recommending books, would it make sense to use Amazon Referral Links? There must be some cough nonprofit we would like to benifet from the referral fees. And I imagine it would not be too hard for the site to automatically add a referral code to links to Amazon submitted in comments.

tall ships down

subject: the final voyages and sinkings of five large sailing vessels

big lessons: 1) if you have a weakness (personally, in your competence or temperament, or structurally, in your vessel or equipment) and spend enough time in an unpredictable environment, it will eventually be exploited. 2) the fact that an environment is unpredictable does not relieve you from the responsibility of considering risks and working to minimize them. 3) we often have an inkling about our weaknesses, but if we've gotten by so far without major incident, we see no pressing need to address them. 4) if you're the captain/leader of an operation, know what the hell you're doing. if you're the equivalent of an ordinary seaman, make it a priority to become competent enough to identify a leader who doesn't know what the hell s/he's doing.

Legacy of Ashes: The History of the CIA by Tim Weiner

Summary: A history of the CIA from its origins in US military intelligence units up to right after 9/11. Based only on primary sources (released CIA documents, interviews with insides, etc.; no rumors or unconfirmed theories), Weiner shows how the CIA has routinely failed to meet its goals and has often made bad situations worse. All of this despite filling the organization with intelligent men ready to die to defend their country by providing military intelligence to the president and engaging in covert operations.

Lessons: There are many small lessons you can learn from the episodes of the history, but the big lesson you'll find is that when an organization has a culture of secrecy, to the point that not even the people in charge with the highest clearance have access to all the secrets, you open the door for mismanagement and abuse. But because the CIA was designed to be opaque, even to the president, it had no accountability or oversight, became inbred with bad ideas, and began to operate independently of US policy by receiving money from Congress and hiding its operations, even from units within itself. The CIA is a case study of the tremendous damage that a few men with power, money, and secrecy can do, where only luck prevented what could have been disastrous consequences.

A Demon Of Our Own Design by Richard Bookstaber

This book came out two years ago but reads like it was just written.

The lesson is similar to e.g. When Genius Failed but it gets into the grit a bit more and is more detailedly insightful about markets. Bookstaber has worked in markets rather than just writing about them, and it shows.

Dull in spots, but that's fairly standard in books like this, since you can't sell a 130-page book.

The March of Folly: From Troy to Vietnam by Barbara Tuchman.

Fumbling the Future: How Xerox Invented, then Ignored, the First Personal Computer

What Made Gertie Gallop? : Learning From Project Failures by O. P. Kharbanda and Jeffrey K. Pinto

The projects reviewed here are old enough that they have been analyzed well enough for fairly complete understanding to be possible. The mega-scale of the projects makes them less than directly applicable for most readers, but their large scale makes for a completeness in their management, smaller projects frequently skimp on their formal management, and more thorough documentation, that makes for a better analysis.

The failures are well illustrated by the projects chosen and the writing does not get in the way of the analyses. This book is very clearly written, the individual project analyses can almost be read like short stories, but with the added benefit of being factual. There are all sorts of failure modes discussed, failures of structural understanding, failures of planning, of political support, of lack of market demand, and more.

The Best and the Brightest (1972) - David Halberstam

Amazon Wikipdia

from Wikipedia: The Best and the Brightest (1972) is an account by journalist David Halberstam of the origins of the Vietnam War. The focus of the book is on the foreign policy crafted by the academics and intellectuals who were in John F. Kennedy's administration, and the consequences of those policies in Vietnam.

One distinction made by the book is the difference between acting based on reasoning or based on principles. The men acting based on reasoning got good results, but in times of high stress or confusion they made bad decisions.

Albert Speer's autobiography, Inside the Third Reich.

Under Hitler's guidance, Speer pioneered the "ruin value" ideology in architecture - thus in a way focusing his trade on the question of how to fail with magnificence.

His attempts to build in a Berlin of the distant future an exponential replication of the ruins of the Roman empire was either never actualized or bulldozed by the allies. Today all that remains of his attempt to harness failure is a row of lampposts on a side street in Berlin.

Judgement Day by Nathaniel Branden. (I think this is the same book as My Years With Ayn Rand.)

Hero becomes Ayn Rand's closest confidant, co-builder of Objectivism, lover; gets drummed out.

Message: every cause wants to be a cult; the enormous power of personal charisma; an excellent antidote for the recalcitrant Objectivist in your life.

There is a short list here: http://soobdujour.blogspot.com/2001/10/failure-reading-list.html

What are the best books about the fall of the Roman Empire?

This seems like one of the most important failures in history, and most important to understand.

I just started Peter Heather's book, but haven't read enough to say anything except that the writing is a bit clunky.

The Titanic is a famous story of failure. I don't know what books to read about it, though, or what lessons to learn from it that we don't already know.

In a very concise and abstract form: http://despair.com/mis24x30prin.html