PhilGoetz comments on Reason as memetic immune disorder - Less Wrong

215 Post author: PhilGoetz 19 September 2009 09:05PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (166)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 20 September 2009 03:01:01AM 2 points [-]

the only safe AI would be a logic system using a consistent logic, so that we could verify that certain undesirable statements were false in that system

Could be correct or wildly incorrect, depending on exactly what he meant by it. Of course you have to delete "the only", but I'd be pretty doubtful of any humans trying to do recursive self-modification in a way that didn't involve logical proof of correctness to start with.

Comment author: PhilGoetz 20 September 2009 09:30:00AM *  8 points [-]

One of the big problems is that he was trying to talk about the logical correctness of human-level symbolic statements about the world. Even if the logic is correct, there is no correct, consistent mapping from the analog world, to symbolic descriptions, and back. A mapping that's close enough to work 99.99% of the time isn't good enough when you're talking about proof.