I shared one quote here. More from Life at the Speed of Light:
Safety, of course, is paramount. The good news is that, thanks to a debate that dates back to Asilomar in the 1970s, robust and diverse regulations for the safe use of biotechnology and recombinant-DNA technology are already firmly in place. However, we must be vigilant and never drop our guard. In years to come it might be difficult to identify agents of concern if they look like nothing we have encountered before. The political, societal, and scientific backdrop is continually evolving and has shifted a great deal since the days of Asilomar. Synthetic biology also relies on the skills of scientists who have little experience in biology, such as mathematicians and electrical engineers. As shown by the efforts of the budding synthetic biologists at iGEM, the field is no longer the province of highly skilled senior scientists only. The democratization of knowledge and the rise of “open-source biology”; the establishment of a biological design-build facility, BIOFAB in California; and the availability of kitchen-sink versions of key laboratory tools, such as the DNA-copying method PCR, make it easier for anyone — including those outside the usual networks of government, commercial, and university laboratories and the culture of responsible training and biosecurity — to play with the software of life.
There are also “biohackers” who want to experiment freely with the software of life. The theoretical physicist and mathematician Freeman Dyson has already speculated on what would happen if the tools of genetic modification became widely accessible in the form of domesticated biotechnology: “There will be do-it-yourself kits for gardeners who will use genetic engineering to breed new varieties of roses and orchids. Also kits for lovers of pigeons and parrots and lizards and snakes to breed new varieties of pets. Breeders of dogs and cats will have their kits too.”
Many have focused on the risks of this technology’s falling into the “wrong hands.” The events of September 11, 2001, the anthrax attacks that followed, and the H1N1 and H7N9 influenza pandemic threat have all underscored the need to take their concerns seriously. Bioterrorism is becoming ever more likely as the technology matures and becomes ever more available. However, it is not easy to synthesize a virus, let alone one that is virulent or infective, or to create it in a form that can be used in a practical way as a weapon. And, of course, as demonstrated by the remarkable speed with which we can now sequence a pathogen, the same technology makes it easier to counteract with new vaccines.
For me, a concern is “bioerror”: the fallout that could occur as the result of DNA manipulation by a non-scientifically trained biohacker or “biopunk.” As the technology becomes more widespread and the risks increase, our notions of harm are changing, along with our view of what we mean by the “natural environment” as human activities alter the climate and, in turn, change our world.
In a similar vein, creatures that are not “normal” tend to be seen as monsters, as the product of an abuse of power and responsibility, as most vividly illustrated by the story of Frankenstein. Still, it is important to maintain our sense of perspective and of balance. Despite the knee-jerk demands for ever more onerous regulation and control measures consistent with the “precautionary principle” — whatever we mean by that much-abused term — we must not lose sight of the extraordinary power of this technology to bring about positive benefits for the world.
Also from Life at the Speed of Light:
......the Presidential Commission for the Study of Bioethical Issues released a report in December 2010 entitled New Directions: The Ethics of Synthetic Biology and Emerging Technologies...
Among its recommendations to the president, the commission said that the government should undertake a coordinated evaluation of public funding for synthetic-biology research, including studies on techniques for risk assessment and risk reduction and on ethical and social issues, so as to reveal noticeable gaps, if one considered that
One open question in AI risk strategy is: Can we trust the world's elite decision-makers (hereafter "elites") to navigate the creation of human-level AI (and beyond) just fine, without the kinds of special efforts that e.g. Bostrom and Yudkowsky think are needed?
Some reasons for concern include:
But if you were trying to argue for hope, you might argue along these lines (presented for the sake of argument; I don't actually endorse this argument):
The basic structure of this 'argument for hope' is due to Carl Shulman, though he doesn't necessarily endorse the details. (Also, it's just a rough argument, and as stated is not deductively valid.)
Personally, I am not very comforted by this argument because:
Obviously, there's a lot more for me to spell out here, and some of it may be unclear. The reason I'm posting these thoughts in such a rough state is so that MIRI can get some help on our research into this question.
In particular, I'd like to know: