= 27d567bc2d48d9f40e9ccc20ea370b15)
XiXiDu comments on Tools want to become agents - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (81)
How do you decide at what point your grasp of a hypothetical system is sufficient, and the probability that it will be build large and robust enough, for it to make sense to start thinking about hypothetical failure modes?
? Explain. I can certainly come up with two hypothetical AI designs, call one a tool and the other an agent (and expect that almost everyone would agree with this, because tool vs agent is clearer at the extremities than in the middle), set up a toy situation, and note that the tools top plan is to make itself into the agent design. The "tool wants to be agent" is certainly true, in this toy setup.
The real question is how much this toy example generalises to real-world scenarios, which is a much longer project. Daniel Dewey has been doing some general work in that area.
My perception, possibly misperception, is that you are too focused on vague hypotheticals. I believe that it is not unlikely that future tool AI will be based on, or be inspired by (at least partly), previous generations of tool AI that did not turn themselves into agent AIs. I further believe that, instead of speculating about specific failure modes, it would be fruitful to research whether we should expect some sort of black swan event in the development of these systems.
I think the idea around here is to expect a strong discontinuity and almost completely dismiss current narrow AI systems. But this seems like black-and-white thinking to me. I don't think that current narrow AI systems are very similar to your hypothetical superintelligent tools. But I also don't think that it is warranted to dismiss the possibility that we will arrive at those superintelligent tools by incremental improvements of our current systems.
What I am trying to communicate is that it seems much more important to me to technically define at what point you believe tools to turn into agents, rather than using it as a premise for speculative scenarios.
Another point I would like to make is that researching how to create the kind of tool AI you have in mind, and speculating about its failure modes, are completely intervened problems. It seems futile to come up with vague scenarios of how these completely undefined systems might fail, and to expect to gain valuable insights from these speculations.
I also think that it would make sense to talk about this with experts outside of your social circles. Do they believe that your speculations are worthwhile at this point in time? If not, why not?
Just because I haven't posted on this, doesn't mean I haven't been working on it :-) but the work is not yet ready.
That's exactly what the plan is now: I think I have enough technical results that I can start talking to the AI and AGI designers.
I'm curious - who are the AI and AGI designers- seeing one hasn't been publicly built yet. Or is this other researchers in the AGI field. If you are looking for feedback from a technical though not academic, I'd be very interested in assisting.
There are a half-dozen AGI projects with working implementations. There are multiple annual conferences where people working on AGI share their results. There's literature on the subject going back decades, really to the birth of AI in the 50's and 60's. The term AGI itself was coined by people working in this field to describe what they are building. Maybe you mean something different than AGI when say "one hasn't been publicly built yet" ?
There seems to be some serious miscommunication going on here. By "AGI", do you mean a being capable of a wide variety of cognitive tasks, including passing the Turing Test? By "AGI project", do you mean an actual AGI, and not just a project with AGI as its goal? By "working implementation", do you mean actually achieving AGI, or just achieving some milestone on the way?
I meant Artificial General Intelligence as that term has been first coined and used in the AI community: the ability to adapt to any new environment or task.
Google's machine learning algorithms can not just correctly classify videos of cats, but can innovate the concept of a cat given a library of images extracted from video content, and no prior knowledge or supervisory feedback.
Roomba interacts with its environment to build a virtual model of my apartment, and uses that acquired knowledge to efficiently vacuum my floors while improvising in the face of unexpected obstacles like a 8mo baby or my cat.
These are both prime examples of applied AI in the marketplace today. But ask Google's neural net to vacuum my floor, or a Roomba to point out videos of cats on the internet and ... well the hypothetical doesn't even make sense -- there is an inferential gap here that can't be crossed as the software is incapable of adapting itself.
A software program which can make changes to its own source code -- either by introspection or random mutation -- can eventually adapt to whatever new environment or goal is presented to it (so long as the search process doesn't get stuck on local maxima, but that's a software engineering problem). Such software is Artificial General Intelligence, AGI.
OpenCog right now has a rather advanced evolutionary search over program space at its core. On youtube you can find some cool videos of OpenCog agents learning and accomplishing arbitrary goals in unstructured virtual environments. Because of the unconstrained evolutionary search over program space, this is technically an AGI. You could put it in any environment with any effectors and any goal and eventually it would figure out both how that goal maps to the environment and how to accomplish it. CogPrime, the theoretical architecture OpenCog is moving towards, is "merely" an addition of many, many other special-purpose memory and heuristic components which both speed the process along and make the agent's thinking process more human-like.
Notice there is nothing in here about the Turing test, nor should there be. Nor is there any requirement that the intelligence be human-level in any way, just that it could be given enough processing power and time. Such intelligences already exist.
"Pass the Turing Test" is a goal, and is therefore a subset of GI. The Wikipedia article says "Artificial general intelligence (AGI) is the intelligence of a (hypothetical) machine that could successfully perform any intellectual task that a human being can."
Your claim that OpenCog can "eventually" accomplish any task is unsupported, is not something that has been "implemented", and is not what is generally understood as what AGI refers to.
That quote describes what a general intelligence can do, not what it is. And you can't extract the Turing test from it. A general intelligence might perform tasks better but in a different way which distinguishes it from a human.
I explained quite well how OpenCog's use of MOSES -- already implemented -- to search program space achieves universality. It is your claim that OpenCog can't accomplish (certain?) tasks that is unsupported. Care to explain?
Can you list the 6 working AGI projects - I'd be interested but I suspect we are talking about different things.
OpenCog, NARS, LIDA, Soar, ACT-R, MicroPsi. More:
http://wiki.opencog.org/w/AGI_Projects http://bicasociety.org/cogarch/architectures.htm
Not sure yet - taking advice. The AI people are narrow AI developers, and the AGI people are those that are actually planning to build an AGI (eg Ben Goertzl).
For a very different perspective from both narrow AI and to a lesser extent Goertzel*, you might want to contact Pat Langley. He is taking a Good Old-Fashioned approach to Artificial General Intelligence:
http://www.isle.org/~langley/
His competing AGI conference series:
http://www.cogsys.org/
EDIT: Not that I think you shouldn't be talking to Goertzel! In fact I think his CogPrime architecture is the only fully fleshed out AGI design which as specified could reach and surpass human intelligence, and the GOLUM meta-AGI architecture is the only FAI design I know of. My only critique is that certain aspects of it are cutting corners, e.g. the rule-based PLN probabilistic reasoning engine vs an actual Bayes net updating engine a la Pearl et al.
Thanks!
It would be helpful if you spelt out your toy situation, since my intuition are currently running in the opposite direction.