Peter Thiel: So you’re both thinking it will all fundamentally work out.
Scott Brown: Yes, but not in a wishful thinking way. We need to treat our work with the reverence you’d give to building bombs or super-viruses. At the same time, I don’t think hard takeoff scenarios like Skynet are likely. We’ll start with big gains in a few areas, society will adjust, and the process will repeat.
I don't believe anyone working on AI is actually treating it that way. I do hope, however, that whenever there are signs of a possible breakthrough, researchers will stop, assess what they have very carefully, and build a lot of safety features before doing any more development. Most important of all, I hope that whoever makes the key discoveries does not publish their results in a way that would enable more reckless groups to copy them
I expect the military and megacorps will be the biggest advocates of closed source machine intelligence software.
That's what happened when the government tried to keep cryptography out of citizens' hands, anyway.
Such efforts are ultimately futile, but they do sometimes act to slow progress down - thereby helping those with early access attain their own goals.
http://blakemasters.tumblr.com/post/24464587112/peter-thiels-cs183-startup-class-17-deep-thought
Some perspectives on AI risk that might be interesting. Peter is (the primary?) donor for SI, and an investor in AGI startup Vicarious Systems.