= 27d567bc2d48d9f40e9ccc20ea370b15)
billswift comments on Open Thread: September 2009 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (179)
Such a design would be harder to reason about.
Let's say you've got a prototype you want to improve. How do you tell if a proposed change would make it smarter, break it, introduce a subtle cognitive bias, or make the AI want to kill you?
In order to set on limits on the kinds of things an AI will do, you need to understand how it works. You can't be experimenting on a structure you partially understand, AND be certain that the experiments won't be fatal.
This is easier when you've got a clearly defined structure to the AI, and know how the parts interact, and why.
In other words they are doing it where the light's better, rather than where they dropped the keys. Given the track record of correctness proofs in comp sci, I don't think provably Friendly AI is even possible, hopefully I'm wrong there, but all they are doing is further crippling their likelihood of achieving AI before some military or business does.