Strange7 comments on What can you do with an Unfriendly AI? - Less Wrong

16 Post author: paulfchristiano 20 December 2010 08:28PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (127)

You are viewing a single comment's thread. Show more comments above.

Comment author: Strange7 22 December 2010 08:47:26AM 0 points [-]

Futile brute-force testing is preferred because so many of the people involved are unwilling or unable to adequately formalize their actual requirements.

Comment author: paulfchristiano 22 December 2010 08:52:14AM 1 point [-]

It is technically true that the people involved are unable to adequately formalize their actual requirements, but that is because precisely formalizing their requirements is a problem much harder than human mathematicians or computer scientists have ever solved. All humans would be "unwilling or unable to adequately formalize their actual requirements." Notice, for example, that mathematicians can't precisely formalize modern mathematics in the required sense either, and that problem is still much easier than formal verification for the sorts of things modern commercial projects are trying to accomplish.