You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Adele_L comments on Probabilistic Löb theorem - Less Wrong Discussion

24 Post author: Stuart_Armstrong 26 April 2013 06:45PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (39)

You are viewing a single comment's thread.

Comment author: Adele_L 28 April 2013 03:32:12AM 0 points [-]

So assuming that the Löb's theorem problem is actually solved, the next step is to define an ideal AI which can self modify without weakening its mathematical system? Is it clear how to do this once the Löb obstruction is dealt with, or is this another very difficult math problem?

Comment author: Stuart_Armstrong 28 April 2013 07:32:07AM 0 points [-]

Nope, it's pretty much done - you can trust your future copies probabilistically, without weakening. However, that would require a P that's both defined and computable - not a trivial task.