eli_sennesh comments on Will AGI surprise the world? - Less Wrong

12 Post author: lukeprog 21 June 2014 10:27PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (129)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 22 June 2014 04:36:13PM 1 point [-]

"Having a machine write code at the level of a human programmer" is a strawman. One can already think about machine learning techniques as the computer writing its own classification programs. These machines already "write code" (classifiers) better than any human could under the same circumstances.. it just doesn't look like code a human would write.

A significant pieces of my own architecture is basically doing the same thing but with the classifiers themselves composed in a nearly turing-complete total functional language, which are then operated on by other reflective agents who are able to reason about the code due to its strong type system. This isn't the way humans write code, and it doesn't produce an output which looks like "source code" as we know it. But it does result in programs writing programs faster, better, and cheaper than humans writing those same programs.

Regarding what AGI is "about", yes that is true in the strictest, definitional sense. But what I was trying to convey is how AGI is separate from narrow AI in that it is basically a field of meta-AI. An AGI approaches a problem by first thinking about how to solve the problem. It first thinks about thinking, before it thinks.

And yes, there are generally multiple ways it can actually accomplish that, e.g. the AGI could not actually solve the problem or modify itself to solve the problem, but instead output the source code for a narrow AI which efficiently does so. But if you draw the system boundary large enough, it's effectively the same thing.

Comment author: [deleted] 25 June 2014 09:12:09AM *  0 points [-]

A significant pieces of my own architecture is basically doing the same thing but with the classifiers themselves composed in a nearly turing-complete total functional language, which are then operated on by other reflective agents who are able to reason about the code due to its strong type system.

Hmmm... Do you have a completeness result? I mean, I can see that if you make it a total language, you can just use coinduction to reason about indefinite computing processes, but I'm wondering what sort of internal logic you're using that would allow complete reasoning over programs in the language and decidable typing (since to have the agent rewrite its own code it will also have to type-check its own code).

Current theorem-proving systems like Coq that work in logics this advanced usually have undecidable type inference somewhere, and require humans to add type annotations sometimes.