lukeprog comments on Tiling Agents for Self-Modifying AI (OPFAI #2) - Less Wrong

55 Post author: Eliezer_Yudkowsky 06 June 2013 08:24PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (260)

You are viewing a single comment's thread.

Comment author: lukeprog 10 January 2014 04:39:28PM 2 points [-]

if you can't even answer the kinds of simple problems posed within the paper... then you must be very far off from being able to build a stable self-modifying AI.

For the record, I personally think this statement is too strong. Instead, I would say something more like what Paul said:

Working on [the Lobian obstacle] directly may be less productive than just trying to understand how reflective reasoning works in general---indeed, folks around here definitely try to understand how reflective reasoning works much more broadly, rather than focusing on this problem. The point of this post is to state a precise problem which existing techniques cannot resolve, because that is a common technique for making progress.