orthonormal comments on AI cooperation in practice - Less Wrong

26 Post author: cousin_it 30 July 2010 04:21PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (157)

You are viewing a single comment's thread. Show more comments above.

Comment author: rwallace 31 July 2010 03:17:03AM 0 points [-]

No, I mean the obvious implementation would have the program basically asking "what does this program return? Well, it tries to find out what this program returns. What does this program return? Well, it tries to find out..." until it hits the length limit and comes back with no solution found (or, if the length limit is lifted, keeps going forever). This is true even if the proof checker always returns quickly.

Comment author: orthonormal 31 July 2010 06:35:55PM 0 points [-]

This isn't the same as the calculator that asks itself what value it will return, because there's a subtle separation of levels here.