FAWS comments on What a practical plan for Friendly AI looks like - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (13)
Simplicity has its value, but one shouldn't pursue plans any simpler than the simplest plan that might actually work. We don't know whether it's even in principle possible for us to knowingly construct an explicit goal system that matches our values and doesn't offer any room for a super-human AI to cleverly "optimize" in unexpected ways without referencing our values themselves. But I would be extremely skeptical of any attempt inspired by contract law. In your analogy this plan sounds more like "balloon that goes really high" to me. Perhaps the Singinst is pursuing Verneian cannons and no one has thought of rockets yet.