xnn comments on Naturalistic trust among AIs: The parable of the thesis advisor's theorem - Less Wrong

24 Post author: Benja 15 December 2013 08:32AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (20)

You are viewing a single comment's thread. Show more comments above.

Comment author: xnn 15 December 2013 08:21:50PM 2 points [-]

"therefore, any serious AI needs a formal system with more oomph than PA"

The problem with that is that the same argument goes through in exactly the same way with any stronger system replacing PA. You might first try something like adding a rule "if PA proves that PA proves S, then S". This solves your original problem, but introduces new ones: there are now new statements that your system can prove that it can prove, but that it can't prove. Eliezer discusses this system, under the name PA+1, in You Provably Can't Trust Yourself .