You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Mitchell_Porter comments on Supposing you inherited an AI project... - Less Wrong Discussion

-5 Post author: bokov 04 September 2013 08:07AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (23)

You are viewing a single comment's thread. Show more comments above.

Comment author: Mitchell_Porter 05 September 2013 09:18:44AM 0 points [-]

You're right, in the sense that there's nothing here about how to generate accurate representations of the world. According to A.2, the user provides the representations. But even if the program is just a numerical optimizer, it's a powerful one, because it's supposed to be able to optimize an arbitrary function (arbitrary network of nodes, as represented in the script).

So it's as if the unfinished AI project already has the part of the code that will do the heavy lifting when problems are solved, and what remains to be done - which is still both important and difficult - is everything that involves transmitting intentions correctly to this AI core, and ensuring that all that raw power isn't used in the service of the wrong goals.