You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

g_pepper comments on Computation complexity of AGI design - Less Wrong Discussion

6 Post author: Squark 02 February 2015 08:05PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (69)
Tags: anthropics agi ai-risk evolution ai-foom

Comments (69)

You are viewing a single comment's thread.

Comment author: g_pepper 03 February 2015 01:32:08AM 3 points [-]

I have doubts regarding this:

It feels like there is a principle of mathematics which rules out algorithms that are "too good to be true": a single "magic wand" to solve all problems. In a similar way, AGI is a "magic wand": it solves "all" problems because you can simply delegate them to the AGI.

It seems to me that either human-level GI is impossible or it is not. But, we humans constitute an existence proof that it is not impossible. Therefore, Gödel does not apply; clearly human-level GI is not "too good to be true". And, Gödel's incompleteness theorems really have nothing to do with how hard it might be to come up with a solution to a computable problem.

I also have doubts regarding this:

This way the AGI design problem becomes an optimization problem: find a program with an intelligence metric as high as possible. The NP-connection now suggests the following conjecture: the AGI optimization program is of exponential complexity in program length. Of course we don't necessarily need the best program of a given length...

The bold portion of the above section is key - there are various NP-hard optimization problems for which good approximate solutions can be found in polynomial time. Since human or super-human AGI does not require a maximally intelligent program, even if you could show that the AGI optimization problem is NP-hard that would say nothing about the difficulty of finding a human or super-human level AGI.

Comment author: Squark 03 February 2015 06:33:57AM 1 point [-]

It seems to me that either human-level GI is impossible or it is not. But, we humans constitute an existence proof that it is not impossible.

Of course. This is why I'm not saying human-level GI is impossible. I'm saying that designing it from scratch is impossible.

And, Gödel's incompleteness theorems really have nothing to do with how hard it might be to come up with a solution to a computable problem.

My argument is not in any sense a proof from Goedel incompleteness. Goedel incompleteness is just a suggestive analogy.

...there are various NP-hard optimization problems for which good approximate solutions can be found in polynomial time. Since human or super-human AGI does not require a maximally intelligent program, even if you could show that the AGI optimization problem is NP-hard that would say nothing about the difficulty of finding a human or super-human level AGI.

I mostly agree: except for the "nothing" part. I think it would definitely cause me to update towards "designing AGI is infeasible". I hinted at a possible relation between intelligence and LK-complexity. If such a relation can be proven and human intelligence can be estimated we would have a more definite answer.

Powered by Reddit Powered by Reddit