= 27d567bc2d48d9f40e9ccc20ea370b15)
Viliam_Bur comments on Reply to Holden on 'Tool AI' - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (348)
Commentary (there will be a lot of "to me"s because I have been a bystander to this exchange so far):
I think this post misunderstands Holden's point, because it looks like it's still talking about agents. Tool AI, to me, is a decision support system: I tell Google Maps where I will start from and where I will leave from, and it generates a route using its algorithm. Similarly, I could tell Dr. Watson my medical data, and it will supply a diagnosis and a treatment plan that has a high score based on the utility function I provide.
In neither case are the skills of "looking at the equations and determining real-world consequences" that necessary. There are no dark secrets lurking in the soul of A*. Indeed, that might be the heart of the issue: tool AI might be those situations where you can make a network that represents the world, identify two nodes, and call your optimization algorithm of choice to determine the best actions to choose to attempt to make it from the start node to the end node.
Reducing the world to a network is really hard. Determining preferences between outcomes is hard. But Tool AI looks to me like saying "well, the whole world is really too much. I'm just going to deal with planning routes, which is a simple world that I can understand," where the FAI tools aren't that relevant. The network might be out of line with reality, the optimization algorithm might be buggy or clumsy, but the horror stories that keep FAI researchers up at night seem impossible because of the inherently limited scope, and the ability to do dry runs and simulations until the AI's model of reality is trusted enough to give it control.
Now, this requires that AI only be used for things like planning where to put products on shelves, not planning corporate strategy- but if you work from the current stuff up rather than from the God algorithm down, it doesn't look like corporate strategy will be on the table until AI is developed to the point where it could be trusted with that. If someone gave me a black box that spit out plans based on English input, then I wouldn't trust it and I imagine you wouldn't either- but I don't think that's what we're looking at, and I don't know if planning for that scenario is valuable.
It seems to me that SI has discussed Holden's Tool AI idea- when it made the distinction between AI and AGI. Holden seems to me to be asking "well, if AGI is such a tough problem, why even do it?".
Let's be a bit more specific -- that is one important point of the article, that as soon as the "Tool AI" definition becomes more specific, the problems start to appear.
We don't want just a system that finds a route between points A and B. We have Google Maps already. By speaking about AGI we want a system that can answer "any question". (Not literally, but it means a wide range of possible question types.) So we don't need an algorithm to find the shortest way between A and B, but we need an algorithm to answer "any question" (or admit that it cannot find an answer), and of course to answer that question correctly.
So could you be just a bit more specific about the algorithm that provides a correct answer to any question? ("I don't know" is also a correct answer, if the system does not know.) Because that is the moment when the problems become visible.
Don't talk about what the Tool AI doesn't do, say what it does. And with a high probability there will be a problem. Of course until you tell what exactly the Tool AI will do, I can't tell you how exactly that problem will happen.
This is relevant:
Please note that AIXI with outputs connected only to a monitor seems like an instance of the Tool AI.
As I read Holden, and on my proposed way of making "agent" precise, this would be an agent rather than a tool. The crucial thing is that this version of AIXI selects actions on the basis of how well they serve certain goals without user approval. If you had a variation on AIXI that identified the action that would maximize a utility function and displayed the action to a user (where the method of display was not done in an open-ended goal-directed way), that would count as a tool.
Sure, but part of my point is that there are multiple options for a Tool AI definition. The one I prefer is narrow AIs that can answer particular questions well- and so to answer any question, you need a Tool that decides which Tools to call on the question, each of those Tools, and then a Tool that selects which answers to present to the user.
What would be awesome is if we could write an AI that would write those Tools itself. But that requires general intelligence, because it needs to understand the questions to write the Tools. (This is what the Oracle in a box looks like to me.) But that's also really difficult and dangerous, for reasons that we don't need to go over again. Notice Holden's claim- that his Tools don't need to gather data because they've already been supplied with a dataset- couldn't be a reasonable limitation for an Oracle in a box (unless it's a really big box).
I think the discussion would be improved by making more distinctions like that, and trying to identify the risk and reward of particular features. That would be demonstrating what FAI thinkers are good at.