= 27d567bc2d48d9f40e9ccc20ea370b15)
asr comments on Students asked to defend AGI danger update in favor of AGI riskiness - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (38)
Almost certainly, the first time you run the seed AI, it'll crash quickly. I think it's very unlikely that you construct a successful-enough-to-be-dangerous AI without a lot of mentally crippled ones first.
If so then we are all going to die. That is, if you have that level of buggy code then it is absurdly unlikely that the first time the "intelligence" part works at all it works well enough to be friendly. (And that scenario seems likely.)
The first machine intellligences we build will be stupid ones.
By the time smarter ones are under developpment we will have other trustworthy smart machines on hand to help keep the newcomers in check.