From Dartnell's The Knowledge:
many inventions seem obvious in retrospect, but sometimes the time of emergence of a key advance or invention doesn’t appear to have followed any particular scientific discovery or enabling technology... The wheelbarrow, for instance, could have occurred centuries before it actually did — if only someone had thought of it. This may seem like a trivial example, combining the operating principles of the wheel and the lever, but it represents an enormous labor saver, and it didn’t appear in Europe until millennia after the wheel (the first depiction of a wheelbarrow appears in an English manuscript written about 1250 AD).
And:
perhaps the most impressive feat of leapfrogging in history was achieved by Japan in the nineteenth century. During the Tokugawa shogunate, Japan isolated itself for two centuries from the rest of the world, forbidding its citizens to leave or foreigners to enter, and permitting only minimal trade with a select few nations. Contact was reestablished in the most persuasive manner in 1853 when the US Navy arrived in the Bay of Edo (Tokyo) with powerfully weaponized steam-powered warships, far superior to anything possessed by the technologically stagnant Japanese civilization. The shock of realization of this technological disparity triggered the Meiji Restoration. Japan’s previously isolated, technologically backward feudal society was transformed by a series of political, economic, and legal reforms, and foreign experts in science, engineering, and education instructed the nation how to build telegraph and railroad networks, textile mills and factories. Japan industrialized in a matter of decades, and by the time of the Second World War was able to take on the might of the US Navy that had forced this process in the first place.
And:
In our history, both compressor and absorption designs for refrigeration were being developed around the same time, but it is the compressor variety that achieved commercial success and now dominates. This is largely due to encouragement by nascent electricity companies keen to ensure growth in demand for their product. Thus the widespread absence of absorber refrigerators today (except for gas-fueled designs for recreation vehicles, where the ability to run without an electrical supply is paramount) is not due to any intrinsic inferiority of the design itself, but far more due to contingencies of social or economic factors. The only products that become available are those the manufacturer believes can be sold at the highest profit margin, and much of that depends on the infrastructure that already happens to be in place. So the reason that the fridge in your kitchen hums— uses an electric compressor rather than a silent absorption design— has less to do with the technological superiority of that mechanism than with quirks of the socioeconomic environment in the early 1900s, when the solution became “locked in.” A recovering post-apocalyptic society may well take a different trajectory in its development.
And:
Whether your garments are stitched from leather or woven fabric, the next problem is how to attach them securely to your body. Disregarding zippers and velcro as too complex to be fabricated by a rebooting civilization, you’re low on options for easily reversible fastenings. The best low-tech solution never occurred to any of the ancient or classical civilizations, yet is now so ubiquitous it has become seemingly invisible. Astoundingly, the humble button didn’t become common in Europe until the mid-1300s. Indeed, it never was developed by Eastern cultures, and the Japanese were absolutely delighted when they first saw buttons sported by Portuguese traders. Despite the simplicity of its design, the new capability offered by the button is transformative. With an easily manufactured and readily reversible fastening, clothes do not need to be loose-fitting and formless so they can be pulled over the top of your head. Instead, they can be put on and then buttoned up at the front, and can be designed to be snugly fitted and comfortable: a true revolution in fashion.
One open question in AI risk strategy is: Can we trust the world's elite decision-makers (hereafter "elites") to navigate the creation of human-level AI (and beyond) just fine, without the kinds of special efforts that e.g. Bostrom and Yudkowsky think are needed?
Some reasons for concern include:
But if you were trying to argue for hope, you might argue along these lines (presented for the sake of argument; I don't actually endorse this argument):
The basic structure of this 'argument for hope' is due to Carl Shulman, though he doesn't necessarily endorse the details. (Also, it's just a rough argument, and as stated is not deductively valid.)
Personally, I am not very comforted by this argument because:
Obviously, there's a lot more for me to spell out here, and some of it may be unclear. The reason I'm posting these thoughts in such a rough state is so that MIRI can get some help on our research into this question.
In particular, I'd like to know: