More (#2) from David and Goliath:
The stranger Cohn had jumped into the cab with happened to be high up at one of Wall Street’s big brokerage firms. And just that week, the firm had opened a business buying and selling options.
“The guy was running the options business but did not know what an option was,” Cohn went on. He was laughing at the sheer audacity of it all. “I lied to him all the way to the airport. When he said, ‘Do you know what an option is?’ I said, ‘Of course I do, I know everything, I can do anything for you.’ Basically by the time we got out of the taxi, I had his number. He said, ‘Call me Monday.’ I called him Monday, flew back to New York Tuesday or Wednesday, had an interview, and started working the next Monday. In that period of time, I read McMillan’s Options as a Strategic Investment book. It’s like the Bible of options trading.”
It wasn’t easy, of course, since Cohn estimates that on a good day, it takes him six hours to read twenty-two pages. He buried himself in the book, working his way through one word at a time, repeating sentences until he was sure he understood them. When he started at work, he was ready. “I literally stood behind him and said, ‘Buy those, sell those, sell those,’” Cohn said. “I never owned up to him what I did. Or maybe he figured it out, but he didn’t care. I made him tons of money.”
...Today he is the president of Goldman Sachs.
And:
One of the best known case studies in criminology is about what happened in the fall of 1969 when the Montreal police went on strike for sixteen hours. Montreal was—and still is—a world-class city in a country that is considered one of the most law-abiding and stable in the world. So, what happened? Chaos. There were so many bank robberies that day—in broad daylight—that virtually every bank in the city had to close. Looters descended on downtown Montreal, smashing windows. Most shocking of all, a long-standing dispute between the city’s taxi drivers and a local car service called Murray Hill Limousine Service over the right to pick up passengers from the airport exploded into violence, as if the two sides were warring principalities in medieval Europe. The taxi drivers descended on Murray Hill with gasoline bombs. Murray Hill’s security guards opened fire. The taxi drivers then set a bus on fire and sent it crashing through the locked doors of the Murray Hill garage. This is Canada we’re talking about. As soon as the police returned to work, however, order was restored.
One open question in AI risk strategy is: Can we trust the world's elite decision-makers (hereafter "elites") to navigate the creation of human-level AI (and beyond) just fine, without the kinds of special efforts that e.g. Bostrom and Yudkowsky think are needed?
Some reasons for concern include:
But if you were trying to argue for hope, you might argue along these lines (presented for the sake of argument; I don't actually endorse this argument):
The basic structure of this 'argument for hope' is due to Carl Shulman, though he doesn't necessarily endorse the details. (Also, it's just a rough argument, and as stated is not deductively valid.)
Personally, I am not very comforted by this argument because:
Obviously, there's a lot more for me to spell out here, and some of it may be unclear. The reason I'm posting these thoughts in such a rough state is so that MIRI can get some help on our research into this question.
In particular, I'd like to know: