0: Recoverable Catastrophe

An apocalypse is an event that permanently damages the world. This scale is for scenarios that are much worse than any normal disaster. Even if 100 million people die in a war, the rest of the world can eventually rebuild and keep going.

1: Economic Apocalypse

The human carrying capacity of the planet depends on the world's systems of industry, shipping, agriculture, and organizations. If the planet's economic and infrastructural systems were destroyed, then we would have to rely on more local farming, and we could not support as high a population or standard of living. In addition, rebuilding the world economy could be very difficult if the Earth's mineral and fossil fuel resources are already depleted.

2: Communications Apocalypse

If large regions of the Earth become depopulated, or if sufficiently many humans die in the catastrophe, it's possible that regions and continents could be isolated from one another. In this scenario, globalization is reversed by obstacles to long-distance communication and travel. Telecommunications, the internet, and air travel are no longer common. Humans are reduced to multiple, isolated communities.

3: Knowledge Apocalypse

If the loss of human population and institutions is so extreme that a large portion of human cultural or technological knowledge is lost, it could reverse one of the most reliable trends in modern history. Some innovations and scientific models can take millennia to develop from scratch.

4: Human Apocalypse

Even if the human population were to be violently reduced by 90%, it's easy to imagine the survivors slowly resettling the planet, given the resources and opportunity. But a sufficiently extreme transformation of the Earth could drive the human species completely extinct. To many people, this is the worst possible outcome, and any further developments are irrelevant next to the end of human history.

 

5: Biosphere Apocalypse

In some scenarios (such as the physical destruction of the Earth), one can imagine the extinction not just of humans, but of all known life. Only astrophysical and geological phenomena would be left in this region of the universe. In this timeline we are unlikely to be succeeded by any familiar life forms.

6: Galactic Apocalypse

A rare few scenarios have the potential to wipe out not just Earth, but also all nearby space. This usually comes up in discussions of hostile artificial superintelligence, or very destructive chain reactions of exotic matter. However, the nature of cosmic inflation and extraterrestrial intelligence is still unknown, so it's possible that some phenomenon will ultimately interfere with the destruction.

7: Universal Apocalypse

This form of destruction is thankfully exotic. People discuss the loss of all of existence as an effect of topics like false vacuum bubbles, simulationist termination, solipsistic or anthropic observer effects, Boltzmann brain fluctuations, time travel, or religious eschatology.

The goal of this scale is to give a little more resolution to a speculative, unfamiliar space, in the same sense that the Kardashev Scale provides a little terminology to talk about the distant topic of interstellar civilizations. It can be important in x risk conversations to distinguish between disasters and truly worst-case scenarios. Even if some of these scenarios are unlikely or impossible, they are nevertheless discussed, and terminology can be useful to facilitate conversation.

Would you notice if science died?

Science is a big deal. It would be worth knowing if it stalled, regressed, or died out, whether the body of knowledge or the techniques for generating more knowledge. You could practice by reviewing history and looking for times and places where it stumbled. In this exercise you have the advantage of hindsight, but the disadvantage of much less direct access to the raw data of the scientific practice of the time. But regardless of how it compares to the real task, this is practice. This is an opportunity to test theories and methods before committing to them. There is a limited amount of history to practice on, but it’s a lot more than the real event, the present.

Many say that they would notice if science died because engineering would grind to a halt or even regress. What does this heuristic say when applied to history? Does it match other criteria?

Many say that the Greeks were good at science and the Romans at engineering (perhaps also the Han vs the Song). This is not really compatible with the heuristic above. What options do we have to draw a coherent conclusion? Either science did not die, or engineering did not advance, or science is not so necessary for engineering; Either we are bad at judging science from history, or we are bad at judging engineering from history, or engineering is not a good heuristic for judging science. None of these are comforting for our ability to judge the future. The third is simply the rejection of the popular heuristic. The first two are the rejection of the exercise of history. But if we cannot judge history, we have no opportunity to practice. Worse, if we are unable to judge history, the present may be no easier.

One recourse is to posit that the past is difficult because of sparse information and that the future we experience ourselves will be easy to judge. But many people lived through the past; what did they think at the time? In particular, how did the Romans think they compared to the Greeks? Did they think that there was progress or regress? Did they agree with modern hindsight? They thought that the Greeks were good at science. Pop science books by Pliny and Seneca are really accounts of Greek knowledge. Similarly, Varro’s practical book of agriculture is based on dozens of Greek sources. And the Romans were proud of their engineering. Frotinus urged his readers to compared the Roman aqueducts to the idle pyramids and wonders of the Greeks. Maybe he should be discounted for his professional interest. But Pliny describes the Roman aqueducts as the most remarkable achievement in the world in the midst of account of Greek knowledge. Indeed, the modern conventional wisdom is probably simply copied from the Romans. Did the Romans endorse the third claim, that science was a prerequisite to engineering? I do not know. Perhaps the they held that it was necessary, but could be left to Greek slaves.

I think that this example should make people nervous about the heuristic about science and engineering. But people who don’t hold any such heuristic should be even more nervous.

I think I know what the answer is. I think that engineering did regress, but the Romans did not notice. They were too impressed by size, so they made bigger aqueducts, without otherwise improving on Greek techniques; and they failed to copy much other Greek technology. Perhaps the heuristic is fine, but it just passes the buck: how much can you trust your judgement of the state of engineering? On the other hand, I think that science regressed much more than engineering, so I do not think them as coupled as the heuristic suggests.

Would you notice if science died? How would you notice? Have you tried that method against history?

---

Some historical test cases: the transition from Greece to Rome; Han vs Tang vs Song; the Renaissance.

Introduction

A recent survey showed that the LessWrong discussion forums mostly attract readers who are predominantly either atheists or agnostics, and who lean towards the left or far left in politics. As one of the main goals of LessWrong is overcoming bias, I would like to come up with a topic which I think has a high probability of challenging some biases held by at least some members of the community. It's easy to fight against biases when the biases belong to your opponents, but much harder when you yourself might be the one with biases. It's also easy to cherry-pick arguments which prove your beliefs and ignore those which would disprove them. It's also common in such discussions, that the side calling itself rationalist makes exactly the same mistakes they accuse their opponents of doing. Far too often have I seen people (sometimes even Yudkowsky himself) who are very good rationalists but can quickly become irrational and use several fallacies when arguing about history or religion. This most commonly manifests when we take the dumbest and most fundamentalist young Earth creationists as an example, winning easily against them, then claiming that we disproved all arguments ever made by any theist. No, this article will not be about whether God exists or not, or whether any real world religion is fundamentally right or wrong. I strongly discourage any discussion about these two topics.

This article has two main purposes:

1. To show an interesting example where the scientific method can lead to wrong conclusions

2. To overcome a certain specific bias, namely, that the pre-modern Catholic Church was opposed to the concept of the Earth orbiting the Sun with the deliberate purpose of hindering scientific progress and to keep the world in ignorance. I hope this would prove to also be an interesting challenge for your rationality, because it is easy to fight against bias in others, but not so easy to fight against bias on yourselves.

The basis of my claims is that I have read the book written by Galilei himself, and I'm very interested (and not a professional, but well read) in early modern, but especially 16-17^th century history.

 

Geocentrism versus Heliocentrism

I assume every educated person knows the name of Galileo Galilei. I won't waste the space on the site and the time of the readers to present a full biography about his life, there are plenty of on-line resources where you can find more than enough biographic information about him.

The controversy?

What is interesting about him is how many people have severe misconceptions about him. Far too often he is celebrated as the one sane man in an era of ignorance, the sole propagator of science and rationality when the powers of that era suppressed any scientific thought and ridiculed everyone who tried to challenge the accepted theories about the physical world. Some even go as far as claiming that people believed the Earth was flat. Although the flat Earth theory was not propagated at all, it's true that the heliocentric view of the Solar System (the Earth revolving around the Sun) was not yet accepted.

However, the claim that the Church was suppressing evidence about heliocentrism "to maintain its power over the ignorant masses" can be disproved easily:

- The common people didn't go to school where they could have learned about it, and those commoners who did go to school, just learned to read and write, not much more, so they wouldn't care less about what orbits around what. This differs from 20-21^th century fundamentalists who want to teach young Earth creationism in schools - back then in the 17^th century, there would be no classes where either the geocentric or heliocentric views could have been taught to the masses.

- Heliocentrism was not discovered by Galilei. It was first proposed by Nicolaus Copernicus almost 100 years before Galilei. Copernicus didn't have any affairs with the Inquisition. His theories didn't gain wide acceptance, but he and his followers weren't persecuted either.

- Galilei was only sentenced to house arrest, and mostly because of insulting the pope and doing other unwise things. The political climate in 17th century Italy was quite messy, and Galilei did quite a few unfortunate choices regarding his alliances. Actually, Galilei was the one who brought religion into the debate: his opponents were citing Aristotle, not the Bible in their arguments. Galilei, however, wanted to redefine the Scripture based on his (unproven) beliefs, and insisted that he should have the authority to push his own views about how people interpret the Bible. Of course this pissed quite a few people off, and his case was not helped by publicly calling the pope an idiot.

- For a long time Galilei was a good friend of the pope, while holding heliocentric views. So were a couple of other astronomers. The heliocentrism-geocentrism debates were common among astronomers of the day, and were not hindered, but even encouraged by the pope.

- The heliocentrism-geocentrism debate was never an ateism-theism debate. The heliocentrists were committed theists, just like  the defenders of geocentrism. The Church didn't suppress science, but actually funded the research of most scientists.

- The defenders of geocentrism didn't use the Bible as a basis for their claims. They used Aristotle and, for the time being, good scientific reasoning. The heliocentrists were much more prone to use the "God did it" argument when they couldn't defend the gaps in their proofs.

 

The birth of heliocentrism.

By the 16th century, astronomers have plotted the movements of the most important celestial bodies in the sky. Observing the motion of the Sun, the Moon and the stars, it would seem obvious that the Earth is motionless and everything orbits around it. This model (called geocentrism) had only one minor flaw: the planets would sometimes make a loop in their motion, "moving backwards". This required a lot of very complicated formulas to model their motions. Thus, by the virtue of Occam's razor, a theory was born which could better explain the motion of the planets: what if the Earth and everything else orbited around the Sun? However, this new theory (heliocentrism) had a lot of issues, because while it could explain the looping motion of the planets, there were a lot of things which it either couldn't explain, or the geocentric model could explain it much better.

 

The proofs, advantages and disadvantages

The heliocentric view had only a single advantage against the geocentric one: it could describe the motion of the planets by a much simper formula.

However, it had a number of severe problems:

- Gravity. Why do the objects have weight, and why are they all pulled towards the center of the Earth? Why don't objects fall off the Earth on the other side of the planet? Remember, Newton wasn't even born yet! The geocentric view had a very simple explanation, dating back to Aristotle: it is the nature of all objects that they strive towards the center of the world, and the center of the spherical Earth is the center of the world. The heliocentric theory couldn't counter this argument.

- Stellar parallax. If the Earth is not stationary, then the relative position of the stars should change as the Earth orbits the Sun. No such change was observable by the instruments of that time. Only in the first half of the 19^th century did we succeed in measuring it, and only then was the movement of the Earth around the Sun finally proven.

- Galilei tried to used the tides as a proof. The geocentrists argued that the tides are caused by the Moon even if they didn't knew by what mechanisms, but Galilei said that it's just a coincidence, and the tides are not caused by the Moon: just as if we put a barrel of water onto a cart, the water would be still if the cart was stationary and the water would be sloshing around if the cart was pulled by a horse, so are the tides caused by the water sloshing around as the Earth moves. If you read Galilei's book, you will discover quite a number of such silly arguments, and you'll see that Galilei was anything but a rationalist. Instead of changing his views against overwhelming proofs, he used  all possible fallacies to push his view through.

Actually the most interesting author in this topic was Riccioli. If you study his writings you will get definite proof that the heliocentrism-geocentrism debate was handled with scientific accuracy and rationality, and it was not a religious debate at all. He defended geocentrism, and presented 126 arguments in the topic (49 for heliocentrism, 77 against), and only two of them (both for heliocentrism) had any religious connotations, and he stated valid responses against both of them. This means that he, as a rationalist, presented both sides of the debate in a neutral way, and used reasoning instead of appeal to authority or faith in all cases. Actually this was what the pope expected of Galilei, and such a book was what he commissioned from Galilei. Galilei instead wrote a book where he caricatured the pope as a strawman, and instead of presenting arguments for and against both world-views in a neutral way, he wrote a book which can be called anything but scientific.

By the way, Riccioli was a Catholic priest. And a scientist. And, it seems to me, also a rationalist. Studying the works of such people like him, you might want to change your mind if you perceive a conflict between science and religion, which is part of today's public consciousness only because of a small number of very loud religious fundamentalists, helped by some committed atheists trying to suggest that all theists are like them.

Finally, I would like to copy a short summary about this book:

In 1651 the Italian astronomer Giovanni Battista Riccioli published within his Almagestum Novum, a massive 1500 page treatise on astronomy, a discussion of 126 arguments for and against the Copernican hypothesis (49 for, 77 against). A synopsis of each argument is presented here, with discussion and analysis. Seen through Riccioli's 126 arguments, the debate over the Copernican hypothesis appears dynamic and indeed similar to more modern scientific debates. Both sides present good arguments as point and counter-point. Religious arguments play a minor role in the debate; careful, reproducible experiments a major role. To Riccioli, the anti-Copernican arguments carry the greater weight, on the basis of a few key arguments against which the Copernicans have no good response. These include arguments based on telescopic observations of stars, and on the apparent absence of what today would be called "Coriolis Effect" phenomena; both have been overlooked by the historical record (which paints a picture of the 126 arguments that little resembles them). Given the available scientific knowledge in 1651, a geo-heliocentric hypothesis clearly had real strength, but Riccioli presents it as merely the "least absurd" available model - perhaps comparable to the Standard Model in particle physics today - and not as a fully coherent theory. Riccioli's work sheds light on a fascinating piece of the history of astronomy, and highlights the competence of scientists of his time.

The full article can be found under this link. I recommend it to everyone interested in the topic. It shows that geocentrists at that time had real scientific proofs and real experiments regarding their theories, and for most of them the heliocentrists had no meaningful answers.

 

Disclaimers:

- I'm not a Catholic, so I have no reason to defend the historic Catholic church due to "justifying my insecurities" - a very common accusation against someone perceived to be defending theists in a predominantly atheist discussion forum.

- Any discussion about any perceived proofs for or against the existence of God would be off-topic here. I know it's tempting to show off your best proofs against your carefully constructed straw-men yet again, but this is just not the place for it, as it would detract from the main purpose of this article, as summarized in its introduction.

- English is not my native language. Nevertheless, I hope that what I wrote was comprehensive enough to be understandable. If there is any part of my article which you find ambiguous, feel free to ask.

I have great hopes and expectations that the LessWrong community is suitable to discuss such ideas. I have experience with presenting these ideas on other, predominantly atheist internet communities, and most often the reactions was outright flaming, a hurricane of unexplained downvotes, and prejudicial ad hominem attacks based on what affiliations they assumed I was subscribing to. It is common for people to decide whether they believe a claim or not, based solely by whether the claim suits their ideological affiliations or not. The best quality of rationalists, however, should be to be able to change their views when confronted by overwhelming proof, instead of trying to come up with more and more convoluted explanations. In the time I spent in the LessWrong community, I became to respect that the people here can argue in a civil manner, listening to the arguments of others instead of discarding them outright.

 

Related: Son of Low Hanging Fruit, Low Hanging Poop

A post by Gregory Cochran's and Henry Harpending's blog West Hunter.

Marcus Terentius Varro  was called the most learned of the Romans.  But what did he know, and how did he know it? I ask because of this quote, from Rerum rusticarum libri III  (Agricultural Topics in Three Books):

“Especial care should be taken, in locating the steading, to place it at the foot of a wooded hill, where there are broad pastures, and so as to be exposed to the most healthful winds that blow in the region. A steading facing the east has the best situation, as it has the shade in summer and the sun in winter. If you are forced to build on the bank of a river, be careful not to let the steading face the river, as it will be extremely cold in winter, and unwholesome in summer. 2 Precautions must also be taken in the neighbourhood of swamps, both for the reasons given, and because there are bred certain minute creatures which cannot be seen by the eyes, which float in the air and enter the body through the mouth and nose and there cause serious diseases.” “What can I do,” asked Fundanius, “to prevent disease if  I should inherit a farm of that kind?” “Even I can answer that question,” replied Agrius; “sell it for the highest cash price; or if you can’t sell it, abandon it.”

I get the distinct impression that someone (probably someone other than Varro) came up with an approximation of germ theory 1500 years before Girolamo Fracastoro.  But his work was lost.

Everybody knows, or should know, that the vast majority of Classical literature has not been preserved.  Those lost works contained facts and ideas that might have value today – certainly there are topics that we understand much better because of insights from Classical literature. For example,  Reich and Patterson find that some of the Indian castes have existed for something like three thousand years:  this is easier to believe when you consider that Megasthenes wrote about the caste system as early as 300 BC.

We don’t put much effort into recovering lost Classical literature.  But there are ways in which we could push harder – by increased funding for work on the Herculaneum scrolls, or the Oxyrhynchus papyri collection, for example.  Some old-fashioned motivated archaeology might get lucky and find another set of Amarna cuneiform letters, or a new Antikythera  mechanism.

From EY's Facebook page, there were two posts that got me thinking about fiction and how to work it better and make it stronger:

It would have been trivial to fix _Revenge of the Sith_'s inadequate motivation of Anakin's dark turn; have Padme already in the hospital slowly dying as her children come to term, not just some nebulous "visions". (Bonus points if you have Yoda lecture Anakin about the inevitability of death, but I'd understand if they didn't go there.) At the end, Anakin doesn't try to choke Padme; he watches the ship with her fly out of his reach, away from his ability to use his unnatural Sith powers to save her. Now Anakin's motives are 320% more sympathetic and the movie makes 170% more sense. If I'd put some serious work in, I'm pretty sure I could've had the movie audience in tears.

I still feel a sense of genuine puzzlement on how such disastrous writing happens in movies and TV shows. Are the viewers who care about this such a tiny percentage that it's not worth trying to sell to them? Are there really so few writers who could read over the script and see in 30 seconds how to fix something like this? (If option 2 is really the problem and people know it's the problem, I'd happily do it for $10,000 a shot.) Is it Graham's Design Paradox - can Hollywood moguls just not tell the difference between competent writers making such an offer, and fakers who'll take the money and run? Are the producers' egos so grotesque that they can't ask a writer for help? Is there some twisted sense of superiority bound up with believing that the audience is too dumb to care about this kind of thing, even though it looks to me like they do? I don't understand how a >$100M movie ends up with flaws that I could fix at the script stage with 30 seconds of advice.

A helpful key to understanding the art and technique of character in storytelling, is to consider the folk-psychological notion from Internal Family Systems of people being composed of different 'parts' embodying different drives or goals. A shallow character is then a character with only one 'part'.

A good rule of thumb is that to create a 3D character, that person must contain at least two different 2D characters who come into conflict. Contrary to the first thought that crosses your mind, three-dimensional good people are constructed by combining at least two different good people with two different ideals, not by combining a good person and a bad person. Deep sympathetic characters have two sympathetic parts in conflict, not a sympathetic part in conflict with an unsympathetic part. Deep smart characters are created by combining at least two different people who are geniuses.

E.g. HPMOR!Hermione contains both a sensible young girl who tries to keep herself and her friends out of trouble, and a starry-eyed heroine, neither of whom are stupid. (Actually, since HPMOR!Hermione is also the one character who I created as close to her canon self as I could manage - she didn't *need* upgrading - I should credit this one to J. K. Rowling.) (Admittedly, I didn't actually follow that rule deliberately to construct Methods, I figured it out afterward when everyone was praising the characterization and I was like, "Wait, people are calling me a character author now? What the hell did I just do right?")

If instead you try to construct a genius character by having an emotionally impoverished 'genius' part in conflict with a warm nongenius part... ugh. Cliche. Don't write the first thing that pops into your head from watching Star Trek. This is not how real geniuses work. HPMOR!Harry, the primary protagonist, contains so many different people he has to give them names, and none of them are stupid, nor does any one of them contain his emotions set aside in a neat jar; they contain different mixtures of emotions and ideals. Combining two cliche characters won't be enough to build a deep character. Combining two different realistic people in that character's situation works much better. Two is not a limit, it's a minimum, but everyone involved still has to be recognizably the same person when combined.

Closely related is Orson Scott Card's observation that a conflict between Good and Evil can be interesting, but it's often not half as interesting as a conflict between Good and Good. All standard rules about cliches still apply, and a conflict between good and good which you've previously read about and to which the reader can already guess your correct approved answer, cannot carry the story. A good rule of thumb is that if you have a conflict between good and good which you feel unsure about yourself, or which you can remember feeling unsure about, or you're not sure where exactly to draw the line, you can build a story around it. I consider the most successful moral conflict in HPMOR to be the argument between Harry and Dumbledore in Ch. 77 because it almost perfectly divided the readers on who was in the right *and* about whose side the author was taking. (*This* was done by deliberately following Orson Scott Card's rule, not by accident. Likewise _Three Worlds Collide_, though it was only afterward that I realized how much of the praise for that story, which I hadn't dreamed would be considered literarily meritful by serious SF writers, stemmed from the sheer rarity of stories built around genuinely open moral arguments. Orson Scott Card: "Propaganda only works when the reader feels like you've been absolutely fair to other side", and writing about a moral dilemma where *you're* still trying to figure out the answer is an excellent way to achieve this.)

Character shallowness can be a symptom of moral shallowness if it reflects a conflict between Good and Evil drawn along lines too clear to bring two good parts of a good character into conflict. This is why it would've been hard for Lord of the Rings to contain conflicted characters without becoming an entirely different story, though as Robin Hanson has just remarked, LotR is a Mileu story, not a Character story. Conflicts between evil and evil are even shallower than conflicts between good and evil, which is why what passes for 'maturity' in some literature is so uninteresting. There's nothing to choose there, no decision to await with bated breath, just an author showing off their disillusionment as a claim of sophistication.

 

I was wondering if we could apply this process to older fiction, Great Literature that is historically praised, and excellent by its own time's standards, but which, if published by a modern author, would seem substandard or inappropriate in one way or another.

Given our community's propensity for challenging sacred cows, and the unique tool-set available to us, I am sure we could take some great works of the past and turn them into awesome works of the present.

Of course, it doesn't have to be a laboratory where we rewrite the whole damn things. Just proprely-grounded suggestions on how to improve this or that work would be great.

 

P.S. This post is itself a work in progress, and will update and improve as comments come. It's been a long time since I've last posted on LW, so advice is quite welcome. Our work is never over.

 

EDIT: Well, I like that this thread has turned out so lively, but I've got finals to prepare for and I can't afford to keep participating in the discussion to my satisfaction. I'll be back in July, and apologize in advance for being such a poor OP. That said, cheers!

TGGP a frequent commenter at Overcoming Bias (and hence old LessWrong), writes about his thoughts on a book by Joseph Tainter.

I’ve seen Joseph Tainter’s “The Collapse of Complex Societies” recommended in a few different places. Jared Diamond’s book might be one of them, the guest-posts of Captain David Ryan aka “Tony Comstock” for James Fallows at the Atlantic might be another. The sidebar of John Robb’s “Global Guerrillas” blog is the only one I remember with certainty. It’s a not a very long book, and you can get the gist of it from Tainter’s wikipedia page.

Lots of people have found civilizational collapses to be interesting, and Tainter reviews many of their theories while finding them wanting. The “eleven major themes in the explanation of collapse” he lists are depletion/cessation of a vital resource, establishment of a new resource base (which I found too stupid to take seriously even momentarily), insurmountable catastrophe, insufficient response to circumstances (which is almost tautological), other complex societies, intruders, class conflict or elite mismanagement, social dysfunction, mystical factors, chance concatenation of events (almost tautological if you don’t think collapse is predetermined) and economic factors. Like Tainter, I find the “mystical” theories to not really constitute theories at all, although some of the most popular writers on the subject (Spengler, Toynbee, various ancients) are included there. Tainter often contrasts “integrative” (or “functional”) theories on the origin of the state/complexity vs “conflict” theories, and acknowledges that he is more partial toward the former. Unfortunately, most of the latter theorists he lists are Marxists and carry a lot of baggage. The observation that throughout much of history some set of people ruled over others as a result of military victory regardless of any benefit to the subjects (though a Leviathan may happen to have upsides) predates Marx, with Ibn Khaldun being one of the few non-Marxist examples Tainter mentions. That’s not to say Tainter is anti-Marxism, he actually compares Marxist “social science” to Einsteinian physics and Darwinian biology! I suppose there is (or was) just such a heavy representation of Marxists among academic anthropologists and historians that Tainter regards Marxism as somewhat normative, whereas to me it’s something weird and laughable like Holocaust revisionism.

Resource depletion is the reverse of the theory I found so absurd, and (showing there is hope for humanity) it is a much more popular theory. J. Donald Hughes blamed Rome’s collapse in part on deforestation, but W. Groenman van Waateringe some years later provided evidence at the time that cereal pollens declined while forest pollens increased. That of course is not a causal proof, since it is documented that when the empire was declining many agricultural regions became depopulated. Waateringe blames agricultural intensification for increasing the population and thus the demands on agriculture, but to me that just raises the question of why marginal agricultural lands weren’t reclaimed. There actually is an explanation for that depopulation, but like Tainter I’m not going to get to that in a hurry. Tainter finds this theory (like most other theories he rejects) unsatisfactory because complex societies should have leaders who notice the depletion and think of a response. I am reminded of David St. Hubbin’s girlfriend in Spinal Tap who says “It’s just a problem! It get’s solved!” Sometimes a solution is not within a society’s feasible choice-set. Tainter briefly acknowledges that possibility (noting that it would have to be proved, which is difficult given how little information we have about many ancient societies) but spends more time castigating imaginary opponents depicted as claiming societal elites just stood around slack-jawed rather than attempting to deal with the situation. I would call that a strawman, except that Jared Diamond’s “Collapse” bears too much resemblance. He also mentions Richard Wilkinson’s documenting that deforestation spurred development in late/post medieval England, which is really just extra evidence that Hughes was wrong (as had already been mentioned) rather than a broader point against a class of theories. I should acknowledge that Tainter also cites evidence on the differential abandonment of cities and the failure to correlate with expected environmental characteristics, which is just the sort of thing that would later puncture Jared Diamond’s take on the Maya. His point that greed is constant enough (or its variance poorly enough understood) to make it a poor explanation of a variable situation is fair enough, but he can’t dismiss theories of collapse based on mismanagement because by their nature they should keep their society going, even if only out of self-interest. There are basic agency problems that mean one shouldn’t identify the interests of elite (or non-elite, for that matter) actors with that of a larger organization. In an uncertain world it also makes sense to discount the future (you or your dynasty might be replaced, might as well get what you can while you can).

Steve Sailer once critiqued Diamond’s thesis by noting that societies tend to die from homicide rather than suicide, and lumping together two of Tainter’s rejected explanations would make for a very popular theory. Tainter, however, would exclude most cases clearly caused by another complex society because those involve absorbtion rather than collapse (indigenous populations thoroughly devastated by disease before Europeans even arrived would be exceptions). So his question is then why a complex society would succumb to less complex intruders. Sometimes it may not be so easy to disentangle the two scenarios, such as when the persian & eastern roman empires exhausted themselves fighting each other, leaving themselves open to the Muslim invaders exploding out of Arabia (although it was only the persians that succumbed in fairly short order, and Tainter wouldn’t consider that a collapse). Tainter says it is “unsatisfactory [...] that a recurrent process – collapse – is explained by a random variable, by historical accident”. If random numbers for that variable (I’m imagining a stochastic process with a threshold for collapse rather than a binary control variable) are constantly being generated over time, it shouldn’t be that unsatisfactory that they recur throughout history. Tainter does make the legitimate point that elites, with a number of Roman emperors being good examples, have often proved capable of dealing with barbarian intrusions. But there’s no guarantee that they will always be successful. He also wonders why invaders would “destroy those things which repay conquest”. The obvious answer is that, by the second law of thermodynamics, it’s very easy to break things, and that includes during the process of conquering & looting. Some relatively sophisticated barbarians may conquer a territory and leave much of the administrative apparatus intact to rule as before, others may have no particular interest (or competence) in being bound to a territory and collecting scraps of taxes from farmers.

The collapse of Rome is probably the most famous example (at least to westerners) and forms one of his three case studies, paradigmatic of the most complex sort of society to collapse. I find it more enlightening than the others because, and call me a drunk looking under a lamppost if you will, it’s the most well documented. Among the things documented is that the proximate cause of collapse was invasion by various (mostly Germanic) barbarians. That is discussed extensively in Peter Heather’s “Empires and Barbarians” which I have discussed earlier. It’s because I read that one so recently that a few of Tainter’s remarks stuck out. Focusing on the internal soundness of a society and its affordable scale/complexity he writes “The Germanic kingdoms that succeeded Roman rule in the West were more successful at resisting invasions”. If he limited that claim to the particular kingdoms which survived past the dark ages, it would be rather tautological. But if he means germanic kingdoms generally, then it just doesn’t seem to be the case. They got invaded and replaced all the time, we just don’t remember the ones that died out. Part of the reason the Romans had such problems with barbarians is that one group would get invaded by another, and then start moving around and displacing other barbarians. The western Roman empire, in contrast, was able to survive many invasions before the last Roman emperor was toppled. Tainter portrays the formation of the empire as a process in which a territory is able to summon the resources to mobilize a force to conquer more territory to extract its resources, then rinse and repeat in a self-sustaining cycle until it expanded too far to get many marginal returns. There is some truth to that, but it overlooks the non-extractive aspect of Roman rule which increased productivity in conquered territories, thereby making those territories more attractive as a target for raiding. In Heather’s story, barbarian confederations on the border engaged in a process of competitive selection for the strength to hold an attractive position (for reasons of trade, raiding and diplomatic subsidy) and eventually the size and cohesion necessary to survive and settle within Roman territory. Focusing on the internals of the collapsed societies, he overlooks any dynamics occurring within outside societies that could give them the capability of defeating the imperial power. Heather’s account is similar to Peter Turchin’s in “War and Peace and War” except that, like Khaldun, Turchin focuses more on the softening effects of metropolitan decadence that renders old dynasties vulnerable to the hardened asabiya-endowed border marchers.

Tainter’s two other case studies are the Mayan lowland citystates and Chaco canyon cliff-dwellers. The Mayans are less complex (or at least less well-documented, since the conquistadors destroyed many of the remaining documents) and the Chacoans even less so. He also used the Ik as an example of an extremely simple society that collapsed even further below the level of familial organization, but he didn’t discuss it all that much and I’m not sure how reliable primary source Colin Turnbull was (supposedly they hadn’t been hunter-gatherers for centuries when their supposed “livelihood” of hunting was banned). The interesting thing about the Maya is that there were multiple relatively equivalent city-states rather than one dominant hegemon. Tainter includes them as a case study of collapse, even as he states elsewhere that collapse is not an option for “peer polity competition” because the weakening of one peer just invites conquest by another. Also, rather than devoting most of their resources under duress to a standing army (something documented in the Roman case) Tainter discusses the building of monuments as conspicuous consumption to demonstrate how powerful and brutal (per the depictions of torture) the city was, rather similar to the story Diamond tells. I don’t know what kind of evidence we have for the scale of their military expenditures, although we know they did war from time to time. There was no writing whatsoever in Chaco canyon, so we are left with the old archeological standby of potsherds and whatnot. Tainter does make the interesting point that the culture benefitted from uniting different ecological niches, with higher elevation territories having more agricultural productivity in cold wet years while lower elevation ones were more productive in warm dry ones. An economist would say that this diversified portfolio allowed for more consumption smoothing. However, I was confused by Tainter’s argument that as more outlier territories were incorporated diversity and gains from exchange went down. As long as the ratio between high and low places was stable, incorporating more territories should not cause any problems in that respect. Admittedly, this does mean that there are more viable subsets of communities that would be individually stable if they withdrew, which is indeed what he claims happened eventually. But he also seemed to be suggesting that the system overally was degrading its performance, without clearly stating whether an excess of a particular type of environment was upsetting the balance.

Tainter’s theory to explain collapse is declining marginal returns. This is a common concept in economics, but it is normally used to understood how equilibrium can develop. Applied to a society, we would expect the declining returns to territorial expansion or administrative complexity (the former often requiring some degree of the latter) to result in eventual stasis rather than collapse. David Friedman has an interesting paper on how the advantages of taxing trade, land or labor gave rise to different equilibria for the sizes & boundaries of polities during the Roman, medieval and nationalist eras in Europe. In Friedman’s theory, each shift between eras resulted from some exogenous change rather being part of the internal logic of societies. Tainter relates some various interesting bits from C. Northcote Parkinson’s “Parkinson’s Law, and Other Studies in Administration”. For example, while “between 1914 and 1967, the number of capital ships in the British Navy declined by 78.9 percent, the number of officers and enlisted men by 32.9 percent, and the number of dockyard workers by 33.7 percent [...] the number of dockyard officials and clerks increased by 247 percent, and the number of Admiralty officals by 767 percent” (emphasis added). Mencius Moldbug would not be surprised to learn that “between 1935 and 1954 the number of officals in the British Colonial office increased by 447 percent” even though “the empire administered by these officials shrank considerably”. These examples are important because they do not demonstrate an increasingly large requirement of administrators for a marginal increase in size/complexity of an entity to be administered, but paying more for less. Parkinson’s explanation was bureaucratic self-serving, which Tainter rejects because he finds trends of increasing hierarchical specialization in the private sector. But because Tainter fails to distinguish between declining marginal returns (eventually reaching zero at a steady-state) and NEGATIVE returns he doesn’t specify whether the latter occurs in the private sector (though Karl Smith would not be surprised if it does for many publicly owned corporations whose shareholders would be better served by liquidation of assets). The growth of administration in higher education would also count, but as a heavily subsidized non-profit sector I can’t say it would qualify. At one point Tainter acknowledges “In many cases this increased, more costly complexity will yield no increased benefits, at other times the benefits will not be proportionate to costs” (emphasis in the original). This is precisely the question at issue of elite mismanagement or the out-of-control inertia of expanding administrative bureaucracies, but as noted he rejected Parkinson’s theory and mocks the idea of societies as runaway trains as self-evidently absurd. Instead he portrays collapse as a choice which is preferable once marginal returns have declined to a certain point. This didn’t entirely make sense to me, because if a society has accidentally shot part the point of zero marginal returns to one of negative returns, the sensible thing is just to reduce that marginal increase in complexity to return to the steady state with zero marginal returns.

The Roman empire sometimes seemed to behave in such a manner, losing some territory and sticking with a more defensible and adminstrable domain (although in Heather’s account some of the lost territory was among the most agriculturally productive), although Tainter thinks the conquests of Britain & Dacia never paid for themselves. So why the path dependency so that changes are not simply reversible? There could be consumption of a not easily renewable resource, a sort of borrowing from the future that leaves future generations deeper in the red. This could happen with soil deterioration, though Tainter doesn’t discuss that much (odd, despite his focus on societies as means of managing sources of energy). His example of Roman emperors increasingly resorting to the debasing of the currency could count (by Diocletian’s time it collected taxes in kind rather than the currency it had rendered nearly worthless), as well as the selling of imperial land. The larger problem in Rome seemed to be an increasingly large portion of subjects who were citizens (both urban proletariat and squabbling elites) subject to fewer or no taxes, while marginal lands were abandoned by overtaxed farmers. An odd feature of the empire was that election officials had to cover the costs of their own office, and as expenses rose there were fewer wealthy people willing to come forward as candidates, until the position was made hereditary. It became obligatory to farm certain deserted lands, with peasants drafted by local city Senates, and Constantine made soldiery a hereditary profession (which required a number of new laws over time to deal with sons who’d rather not follow that career). Taxation of land was simplistic and did not vary based on its quality or yield, so a farmer of marginal land would often be better off working for the owner of a more productive territory and paying rent than failing to cover the taxes on his own plot. With agricultural labor becoming legally tied to the land, we can see the clear beginnings of serfdom and the manorial system. As mentioned, Tainter views the Roman collapse as a choice (as he does others), although of course accounts from the time were more apt to regard it as unfortunate failure or divine punishment.

Interestingly, the “peer polity competition” that replaced Roman civilization is a situation he regards as invulnerable to collapse as opposed to absorbtion, and by removing that “option” he thinks this made peasants demand democratic representation. He acknowledges that this did not happen in the “Warring States” period of China, and instead the Confucian ideology of governance developed. He suggests “Perhaps participatory governance was simply not possible in ancient societies that were so much larger, demographically and territorially, than the Greek city-states”. Someone should have told James Madison (and I’m not being sarcastic). Interestingly enough, there was a civilization of Greek city states which did collapse, just as we’ve mentioned the lowland Maya doing. These are the Mycenaean Greeks who preceded the Dark Ages of Homer’s time. Their collapse is usually attributed to invasion by Dorian Greeks, but Tainter isn’t convinced there’s enough evidence for the Dorians’ presence. Because “Collapse occurs, and can only occur, in a power vacuum” (emphasis in the original) both the Mycenaean Greek and lowland Maya polities must have experienced simultaneous collapse.

The choice of peasants may be limited to passively withdrawing support and just not working very hard (I’d have more to say on that if I’d read James Scott’s “Weapons of the Weak”), but even then I don’t think it’s a desirable outcome for peasants. I’ve mentioned Heather on the greater productivity of Roman territory, and what do you think happened to the masses when that productivity crashed? My understanding of the current archaelogical consensus is that the population crashed as well. Tainter talks about the malnourished skeletons of the peasantry as evidence for the undesirability of certain degrees of complexity, but we also know that English peasants ate better after so many of their peers died of the bubonic plague (it’s also known that peasants have poorer diets than hunter-gatherers, though in Darwinian terms you definitely want to be a farmer). As in James Scott’s account of highland southeast asia (which I don’t entirely buy) was there much cultural defection of the peasantry to the greener pastures outside civilization? Tainter writes that “In 378 [...] Balkan miners went over en masse to the Visigoths”, and that others wished to be conquered/liberated. It is precisely due to the risk of being conquered that he argues is the reason many societies don’t simply revert to a lower level of complexity “even if marginal returns are unfavorable”, but it’s unclear whether conquest is one of outcomes being factored into those marginal returns.

Few people are going to read this book without speculating on their own complex society’s liability to collapse. John Robb and James Kunstler (along with some others in the “Peak Oil” camp) are going to place a high probability on it, while the Singularitarians have the opposite view. Globalization could mean the entire world is now in a state of “peer polity competition” but modern norms (and economic incentives) against conquest and giving war a chance means “failed states” can keep failing for a long time without someone replacing the bad management. Tainter’s studied societies are also Malthusian agricultural ones, it’s hard to know if the same logic will generalize past the industrial revolution. In modern technological economies the costs and benefits of advances may not be simple increasing or declining curves. Robin Hanson doesn’t even consider nearly free energy (which would very important to Tainter) to be nearly as important as the replacement of most human labor by computers (since the latter takes up so much more of GDP). When Tainter was writing there was still just the slightest possibility of nuclear armageddon, now the most likely candidates for death by complexity are grey goo or an unstoppable manmade pandemic. My two cents are that collapse is unlikely in my lifetime, and that’s for the better considering how much worse things could be.

Part of the series AI Risk and Opportunity: A Strategic Analysis.

(You can leave anonymous feedback on posts in this series here. I alone will read the comments, and may use them to improve past and forthcoming posts in this series.)

Building on the previous post on AI risk history, this post provides an incomplete timeline (up to 1993) of significant novel ideas and arguments related to AI as a potential catastrophic risk. I do not include ideas and arguments concerning only, for example, the possibility of AI (Turing 1950) or attempts to predict its arrival (Bostrom 1998).

As is usually the case, we find that when we look closely at a cluster of ideas, it turns out these ideas did not appear all at once in the minds of a Few Great Men. Instead, they grew and mutated and gave birth to new ideas gradually as they passed from mind to mind over the course of many decades.

 

1863: Machine intelligence as an existential risk to humanity; relinquishment of machine technology recommended. Samuel Butler in Darwin among the machines worries that as we build increasingly sophisticated and autonomous machines, they will achieve greater capability than humans and replace humans as the dominant agents on the planet:

...we are ourselves creating our own successors; we are daily adding to the beauty and delicacy of their physical organisation; we are daily giving them greater power and supplying by all sorts of ingenious contrivances that self-regulating, self-acting power which will be to them what intellect has been to the human race. In the course of ages we shall find ourselves the inferior race... the time will come when the machines will hold the real supremacy over the world and its inhabitants...

Our opinion is that war to the death should be instantly proclaimed against them. Every machine of every sort should be destroyed by the well-wisher of his species. Let there be no exceptions made, no quarter shown...

(See also Butler 1872; Campbell 1932.)

1921: Robots as an existential risk. The Czech play R.U.R. by Karel Capek tells the story of robots which grow in power and intelligence and destroy the entire human race (except for a single survivor).

1947: Fragility & complexity of human values (in the context of machine goal systems); perverse instantiation. Jack Williamson's novelette With Folded Hands (1947) tells the story of a race of machines that, in order to follow the Prime Directive: "to serve and obey and guard men from harm." To obey this rule, the machines interfere with every aspect of human life, and humans who resist are lobotomized. Due to the fragility and complexity of human values (Yudkowsky 2008; Muehlhauser and Helm 2012), the machines' rules of behavior had unintended consequences, manifesting a "perverse instantiation" in the language of Bostrom (forthcoming).

(Also see Asimov 1950, 1957, 1983; Versenyi 1974; Minsky 1984; Yudkowsky 2001, 2011.)

1948-1949: Precursor idea to intelligence explosion. Von Neumann (1948) wrote:

...“complication" on its lower levels is probably degenerative, that is, that every automaton that can produce other automata will only be able to produce less complicated ones. There is, however, a certain minimum level where this degenerative characteristic ceases to be universal. At this point automata which can reproduce themselves, or even construct higher entities, become possible.

 

Von Nuemann (1949) came very close to articulating the idea of intelligence explosion:

There is thus this completely decisive property of complexity, that there exists a critical size below which the process of synthesis is degenerative, but above which the phenomenon of synthesis, if properly arranged, can become explosive, in other words, where syntheses of automata can proceed in such a manner that each automaton will produce other automata which are more complex and of higher potentialities than itself.

1951: Potentially rapid transition from machine intelligence to machine takeover. Turing (1951) described ways that intelligent computers might learn and improve their capabilities, concluding that:

...it seems probable that once the machine thinking method has started, it would not take long to outstrip our feeble powers... At some stage therefore we should have to expect the machines to take control...

1959: Intelligence explosion; the need for human-friendly goals for machine superintelligence. Good (1959) describes what he later (1965) called an "intelligence explosion," a particular mechanism for rapid transition from artificial general intelligence to dangerous machine takeover:

Once a machine is designed that is good enough… it can be put to work designing an even better machine. At this point an "explosion" will clearly occur; all the problems of science and technology will be handed over to machines and it will no longer be necessary for people to work. Whether this will lead to a Utopia or to the extermination of the human race will depend on how the problem is handled by the machines. The important thing will be to give them the aim of serving human beings.

(Also see Good 1962, 1965, 1970; Vinge 1992, 1993; Yudkowsky 2008.)

1966: A military arms race for machine superintelligence could accelerate machine takeover; convergence toward a singleton is likely. Dennis Feltham Jones' 1966 novel Colossus depicted what may be a particularly likely scenario: two world superpowers (the USA and USSR) are in an arms race to develop superintelligent computers, one of which self-improves enough to take control of the planet.

In the same year, Cade (1966) argued the same thing:

 

political leaders on Earth will slowly come to realize... that intelligent machines having superhuman thinking ability can be built. The construction of such machines, even taking into account all the latest developments in computer technology, would call for a major national effort. It is only to be expected that any nation which did put forth the financial and physical effort needed to build and programme such a machine, would also attempt to utilize it to its maximum capacity, which implies that it would be used to make major decisions of national policy. Here is where the awful dilemma arises. Any restriction to the range of data supplied to the machine would limit its ability to make effective political and economic decisions, yet if no such restrictions are placed upon the machine's command of information, then the entire control of the nation would virtually be surrendered to the judgment of the robot.

On the other hand, any major nation which was led by a superior, unemotional intelligence of any kind, would quickly rise to a position of world domination. This by itself is sufficient to guarantee that, sooner or later, the effort to build such an intelligence will be made — if not in the Western world, then elsewhere, where people are more accustomed to iron dictatorships.

...It seems that, in the forseeable future, the major nations of the world will have to face the alternative of surrendering national control to mechanical ministers, or being dominated by other nations which have already done this. Such a process will eventually lead to the domination of the whole Earth by a dictatorship of an unparalleled type — a single supreme central authority.

 

(This last paragraph also argues for convergence toward what Bostrom later called a "singleton.")

(Also see Ellison 1967.)

1970: Proposal for an association that analyzes the implications of machine superintelligence; naive control solutions like "switch off the power" may not work because the superintelligence will outsmart us, thus we must focus on its motivations; possibility of "pointless" optimization by machine superintelligence. Good (1970) argues:

Even if the chance that the ultraintelligent machine will be available [soon] is small, the repercussions would be so enormous, good or bad, that it is not too early to entertain the possibility. In any case by 1980 I hope that the implications and the safeguards will have been thoroughly discussed, and this is my main reason for airing the matter: an association for considering it should be started.

(Also see Bostrom 1997.)

On the idea that naive control solutions like "switch off the power" may not work because the superintelligence will find a way to outsmart us, and thus we must focus our efforts on the superintelligence's motivations, Good writes:

Some people have suggested that in order to prevent the [ultraintelligent machine] from taking over we should be ready to switch of its power supply. But it is not as simple as that because the machine could recommend the appointment of its own operators, it could recommend that they be paid well and it could select older men who would not be worried about losing their jobs. Then it could replace its operators by robots in order to make sure that it is not switched off. Next it could have the neo-Luddites ridiculed by calling them Ludditeniks, and if necessary it would later have them imprisoned or executed. This shows how careful we must be to keep our eye on the "motivation" of the machines, if possible, just as we should with politicians.

(Also see Yudkowsky 2008.)

Good also outlines one possibility for "pointless" goal-optimization by machine superintelligence:

If the machines took over and men became redundant and ultimately extinct, the society of machines would continue in a complex and interesting manner, but it would all apparently be pointless because there would be no one there to be interested. If machines cannot be conscious there would be only a zombie world. This would perhaps not be as bad as in many human societies where most people have lived in misery and degradation while a few have lived in pomp and luxury. It seems to me that the utility of such societies has been negative (while in the condition described) whereas the utility of a zombie society would be zero and hence preferable.

(Also see Bostrom 2004; Yudkowsky 2008.)

1974: We can't much predict what will happen after the creation of machine superintelligence. Julius Lukasiewicz (1974) writes:

The survival of man may depend on the early construction of an ultraintelligent machine-or the ultraintelligent machine may take over and render the human race redundant or develop another form of life. The prospect that a merely intelligent man could ever attempt to predict the impact of an ultraintelligent device is of course unlikely but the temptation to speculate seems irresistible.

(Also see Vinge 1993.)

1977: Self-improving AI could stealthily take over the internet; convergent instrumental goals in AI; the treacherous turn. Though the concept of a self-propagating computer worm was introduced by John Brunner's The Shockwave Rider (1975), Thomas J. Ryan's novel The Adolescence of P-1 (1977) tells the story of an intelligent worm that at first is merely able to learn to hack novel computer systems and use them to propagate itself, but later (1) has novel insights on how to improve its own intelligence, (2) develops convergent instrumental subgoals (see Bostrom 2012) for self-preservation and resource acquisition, and (3) learns the ability to fake its own death so that it can grow its powers in secret and later engage in a "treacherous turn" (see Bostrom forthcoming) against humans.

1982: To design ethical machine superintelligence, we may need to design superintelligence first and then ask it to solve philosophical problems (e.g. including ethics).

Good (1982) writes:

Unfortunately, after 2500 years, the philosophical problems are nowhere near solution. Do we need to solve these philosophical problems before we can design an adequate ethical machine, or is there another approach? One approach that cannot be ruled out is first to produce an ultra-intelligent machine and then ask it to solve philosophical problems.

1988: Even though AI poses an existential threat, we may need to rush toward it so we can use it to mitigate other existential threats. Moravec (1988, p. 100-101) writes:

...intelligent machines... threaten our existence... Machines merely as clever as human beings will have enormous advantages in competitive situations... So why rush headlong into an era of intelligent machines? The answer, I believe, is that we have very little choice, if our culture is to remain viable... The universe is one random event after another. Sooner or later an unstoppable virus deadly to humans will evolve, or a major asteroid will collide with the earth, or the sun will expand, or we will be invaded from the stars, or a black hole will swallow the galaxy. The bigger, more diverse, and competent a culture is, the better it can detect and deal with external dangers. The larger events happen less frequently. By growing rapidly enough, a culture has a finite chance of surviving forever.

1993: Physical confinement is unlikely to constrain superintelligences, for superintelligences will outsmart us. Vinge (1993) writes:

I argue that confinement [of superintelligent machines] is intrinsically impractical. For the case of physical confinement: Imagine yourself confined to your house with only limited data access to the outside, to your masters. If those masters thought at a rate — say — one million times slower than you, there is little doubt that over a period of years (your time) you could come up with "helpful advice" that would incidentally set you free...

After 1993. The extropians mailing list was launched in 1991, and was home to hundreds of discussions in which many important new ideas were proposed — ideas later developed in the public writings of Bostrom, Yudkowsky, Goertzel, and others. Unfortunately, the discussions from before 1998 were private, by agreement among subscribers. The early years of the archive cannot be made public without getting permission from everyone involved — a nearly impossible task. I have, however, collected all posts I could find from 1998 onward and uploaded them here (link fixed 04-03-2012).

I will end this post here. Perhaps in a future post I will extend the timeline past 1993, when interest in the subject became greater and thus the number of new ideas generated per decade rapidly increased.

References

• Asimov (1950). The Evitable Conflict
• Asimov (1957). The Naked Sun
• Asimov (1983). The Robots of Dawn
• Bostrom (1997). Predictions from Philosophy? How philosophers could make themselves useful
• Bostrom (1998). How Long before Superintelligence?
• Bostrom (2004). The Future of Human Evolution
• Bostrom (2012). The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents
• Bostrom (forthcoming). Superintelligence.
• Brunner (1975). The Shockwave Rider
• Butler (1863). Darwin among the machines
• Butler (1872). Erewhon.
• Campbell (1932). The Last Evolution
• Capek (1921). R.U.R.
• Ellison (1967). I Have No Mouth, and I Must Scream
• Good (1959). Speculations on perceptrons and other automata
• Good (1962). The social implications of artificial intelligence
• Good (1965). Speculations Concerning the First Ultraintelligent Machine
• Good (1970). Some future social repercussions of computers
• Jones (1966). Colossus.
• Lukasiewicz (1974). The Ignorance Explosion.
• Minsky (1984). Afterward to Vinge's 'True Names'.
• Moravec (1988). Mind Children: The Future of Robot and Human Intelligence.
• Muehlhauser & Helm (2012). The Singularity and Machine Ethics
• Ryan (1977). The Adolescence of P-1
• Turing (1950). Computing Machinery and Intelligence
• Turing (1951). Intelligent machinery, a heretical theory
• Versenyi (1974). Can robots be moral?
• Vinge (1992). A Fire Upon The Deep.
• Vinge (1993). The Coming Technological Singularity.
• Von Neumann (1948). The general and logical theory of automata.
• Von Neumann (1949). Theory and Organization of Complicated Automata. (Five lectures delivered at the University of Illinois in December, 1949. Reprinted in Papers of John Von Neumann on Computers and Computing Theory.)
• Williamson (1947). With Folded Hands.
• Yudkowsky (2001). Creating Friendly AI.
• Yudkowsky (2008). Artificial Intelligence as a Positive and Negative Factor in Global Risk
• Yudkowsky (2011). Complex value systems are required to realize valuable futures

Yay a new cool post is up on West Hunters blog! It is written by Gregory Cochran and Henry Harpending with whom most LWers are probably already familiar with (particularly this awesome entry). It raises some interesting points on biases in academia.

I was contemplating Conan the Barbarian, and remembered the essay that Robert E. Howard wrote about the  background of those stories – The Hyborian Age.  I think that the flavor of Howard’s pseudo-history is a lot more realistic than the picture of the human past academics preferred over the past few decades.

In Conan’s world, it’s never surprising to find a people that once mixed with some ancient prehuman race.  Happens all the time.  Until very recently, the vast majority of workers in human genetics and paleontology were sure that this never occurred – and only changed their minds when presented with evidence that was both strong (ancient DNA)  and too mathematically sophisticated for them to understand or challenge (D-statistics).

Conan’s history  was shaped by the occasional catastrophe.  Most academics (particularly geologists) don’t like catastrophes, but they have grudgingly come to admit their importance – things like the Thera and Toba eruptions, or the K/T asteroid strike and the Permo-Triassic crisis.

Between the time when the oceans drank Atlantis, and the rise of the sons of Aryas, evolution seems to have run pretty briskly, but without any pronounced direction.  Men devolved into ape-men when the environment pushed in that direction (Flores ?)  and shifted right back when the environment favored speech and tools.  Culture shaped evolution, and evolution shaped culture.  An endogamous caste of snake-worshiping priests evolved in a strange direction.  Although their IQs were considerably higher than average, they remained surprisingly vulnerable to sword-bearing barbarians.

In this world, evolution could happen on a time scale of thousands of years, and there was no magic rule that ensured that the outcome would be the same in every group.  It may not be PC to say it, but Cimmerians were smarter than Picts.

The basic idea of their book "The 10 000 Year Explosion" (LessWrong review, Amazon).

Above all, people in Conan’s world fought. They migrated: they invaded.  There was war before, during, and after civilization.  Völkerwanderungs were a dime a dozen. Conquerors spread.  Sometimes they mixed with the locals, sometimes they replaced them – as when the once dominant Hyborians, overrun by Picts, vanished from the earth, leaving scarcely a trace of their blood in the veins of their conquerors. They must have been U5b.

To be fair,  real physical anthropologists in Howard’s day thought that there had been significant population movements and replacements in Europe, judging from changes in skeletons and skulls that accompanied archeological shifts, as when people turned taller, heavier boned , and brachycephalic just as the Bell-Beaker artifacts show up. But those physical anthropologists lost out to people like Boas – liars.

Perhaps this little old entry is relevant here. ^_^

Given the chance (sufficient lack of information), American anthropologists assumed that the Mayans were peaceful astronomers. Howard would have assumed that they were just another blood-drenched snake cult: who came closer?

Now I’m not saying that Howard got every single tiny little syllable of prehistory right.  Not likely: so far, we haven’t seen any signs of Cthulhu-like visitors, which abound in the Conan stories.  So far. But Howard’s priors were more accurate than those of the pots-not-people archeologists: more accurate than people like Excoffier and  Currat, who assume that there hasn’t been any population replacement in Europe since moderns displaced Neanderthals. More accurate than Chris Stringer,  more accurate than Brian Ferguson.

Most important, Conan, unlike the typical professor, knew what was best in life.

Heh.

Cochran you are such a nerd.

What would the world look like without Hitler? Fiction is generally unequivocal about this: the removal of Hitler makes no difference, the world will still lurch towards a world war through some other path. WWII and the Holocaust are such major, defining events of the twentieth century, that we twist counterfactual events to ensure they still happen.

Against this, some have made the argument that Hitler was essentially sole responsible for WWII and especially for the Holocaust - no Hitler, no war, no extermination camps. The no Holocaust argument is quite solid: the extermination system was expensive, militarily counter-productive, and could only have happened given a leader lacking checks and balance and with an idée fixe that overrode everything else (general European antisemitism allowed the Holocaust, but didn't cause it). The no WWII argument points out that Hitler was both irrational and lucky: he often took great risks, on flimsy evidence, and got away with them. Certainly his decisions in the later, post-Barbarossa period of his reign belie political, military or organisational genius. And it was the height of stupidity to have gone to war, for a half of Poland, with simultaneously the world's greatest empire and what appeared to be the overwhelmingly strong French army. Yes Gamelin, the French commander in chief, did behave like a concussed duckling, and the German army outfought the French - but no-one could have predicted this, and no-one sensible would have counted on it, and hence they wouldn't have risked the war. Hitler wan't sensible, and lucked out.

I am an aspiring historian and I'm very interested in ways to apply Bayesian reasoning to history. When I say "history" I mean the study of history -- as a historian, allowing my map of what has happened in the past to match the territory, and being able to represent more accurately the relative strength of historical evidence for and against various historical models.

I know that historical evidence works quite a bit differently from scientific evidence. But I think that historical evidence is also useful. Historians, in recording and assembling secondary sources, assess the relative strength of evidence (mostly primary sources) with regards to a topic already.  But there must be a way to do it more formally. Shouldn't there be a right answer, just as no two people who are completely rational (and have the same information) should ever disagree?

This is a post (or series of posts) I might write in the future, and I have put a bit of thought into it so far, but I need to do quite a bit more research. Is there anyone interested in reading something on this topic? Has it been done before? Is there anyone who is knowledgeable about how historians treat evidence who might be able to offer some insights?

Background: Apostles' Creed, Tsuyoku Naritai

Related to: A Parable on Obsolete Ideologies

Just something I thought I might add to the annals of cases where someone tries to re-interpret an old religious text to mean something more acceptable to the modern ear, in contradiction to what most people (especially its contemporaries) think the texts mean.  And this is not some random person, but Gene Callahan, who makes sure you understand he holds a doctorate in philosophy, and pretty much makes a career out of defending this and anti-reductionist views in general.  Here's the post:

I believe in one God, the Father Almighty, Maker of heaven and earth...
And in one Lord Jesus Christ, the only-begotten Son of God...
And I believe in the Holy Ghost, the Lord and Giver of Life..."

If I say these words, what do I mean? I am asserting that I have some secret knowledge that others do not? Do I believe these things like I believe it will rain tonight?

No, I asserting that, by meditating on these symbols, I believe I will come to understand better what I now know only through a glass darkly.

I believe that I may understand.

I suggested that this is not what most people mean when they say the Creed, but (surprise) the comment was deleted.

(Yes I know Tsuyoku Naritai is not quite the same as Callahan's interpretation, but it's the closest short LW term for the general idea.)

There was a historical shift in beliefs.

But the one we’re concerned with is that women’s libidos went from being considered as powerful or more so than men’s to being essentially erased. Pre-Renaissance examples of horny ladies abound, from the Greeks onward; make your own list, but do include Chaucer. He’s such fun. This change in attitudes appears to have been religiously motivated, and based on the idea that women are more spiritual and sacred than men, meaning less horny. Again, make your own list of contemporary leftovers of this attitude; there are plenty.

By the 18th century, it was taken as read that a woman who did experience (or at least express) sexual desire was suffering from a disorder. One important 1775 study of the subject linked the problem to “secret pollutions”, i.e. wanking, and (I swear I am not making this up) eating too much chocolate. I guess that’d go a ways toward explaining this advertisement. Women were diagnosed with, treated for, and often operated upon for “nymphomania”, the dread condition that causes a woman to want sex. (Talk to your doctor; you may suffer from it yourself!) And yes, by “operated upon”, I mean clitoridectomy. And yes, that’s fucking appalling.

I find this very odd. How could a major cultural lineage be wrong about something so much a part of ordinary experience?

When I say wrong, I don't necessarily mean that we're right, or the ancients were right, though there's a lot of evidence that the Victorians were wrong.

My favorite theory is that people's amount of desire for sex varies sufficiently that there's enough noise to make it easy to see patterns that aren't there. I leave the possibility open that there was a change (possibly dietary) which affected libido levels differently between men and women.

People are sufficiently punitive about sex that there's going to be lies and misdirection to support the current theory about how people are supposed to be.

I think LWers may be intrigued...

Tim McGrew, author of this excellent annotated bibliography on Bayesian reasoning, recently co-authored with his wife Lydia a Bayesian defense of the resurrection of Jesus. I interviewed Lydia for my podcast, here. Atheist Richard Carrier has leveled some objections to their article, but his objections are weak.

Have at it.

I'm reading a popular science encyclopedia now, particularly chapters about the history of physics. The chapter goes on to evaluate the development of the concept of kinetic energy, starting with Aristotle's (grossly incorrect) explanation of a flying arrow saying that it's kept in motion by the air behind it, and then continuing to medieval impetus theory. Added: The picture below illustrates the trajectory of a flying cannonball as described by Albert of Saxony.

What struck me immediately was how drastically different from observations its predictions were. The earliest impetus theory predicted that a cannonball's trajectory was an angle: first a slanted straight line until the impetus runs out, then a vertical line of freefall. A later development added an intermediate stage, as seen on the picture to the left. At first the impetus was at full force, and would launch the cannonball in a straight line; then it would gradually give way to freefall and curve until the ball would be falling in a straight line.

While this model is closer to reality than the original prediction, I still cannot help but think... How could they deviate from observations so strongly?

Yes, yes, hindsight bias.

But if you launch a stream of water out of a slanted tube or sleeve, even if you know nothing about paraboles, you can observe that the curve it follows in the air is symmetrical. Balls such as those used for games would visibly not produce curves like depicted.

Perhaps the idea of verifying theories with experiments was only beginning to coalesce at that time, but what kind of possible thought process could lead one to publish theories so grossly out of touch with everyday observations, even those that you see without making any explicit experiments? Did the authors think something along the lines of "Well, reality should behave this way, and if it doesn't, it's its own fault"?

>Elisabeth Lloyd: I don’t actually know. I think that it’s at a very problematic intersection of topics. I mean, you’re taking the intersection of human evolution, women, sexuality – once you take that intersection you’re bound to kind of get a disaster. More than that, when evolutionists have looked at this topic, I think that they’ve had quite a few items on their agenda, including telling the story about human origins that bolsters up the family, monogamy, a certain view of female sexuality that’s complimentary to a certain view of male sexuality. And all of those items have been on their agenda and it’s quite visible in their explanations.

>Natasha Mitchell: I guess it’s perplexed people partly, too, because women don’t need an orgasm to become pregnant, and so the question is: well, what’s its purpose? Well, is its purpose to give us pleasure so that we have sex, so that we can become pregnant, according to the classic evolutionary theories?

>Elisabeth Lloyd: The problem is even worse than it appears at first because not only is orgasm not necessary on the female side to become pregnant, there isn’t even any evidence that orgasm makes any difference at all to fertility, or pregnancy rate, or reproductive success. It seems intuitive that a female orgasm would motivate females to engage in intercourse which would naturally lead to more pregnancies or help with bonding or something like that, but the evidence simply doesn’t back that up.

The whole discussion. It backs my theory that using evolution to explain current traits seriously tempts people to make things up.