Do we have a clear idea what we mean when I say agent?
Is a Roomba, the robot vacuum cleaner that adapts to walls, furniture, the rate at which the floor gets dirty, and other things, an agent? I don't think so.
Is an air conditioner with a thermostat which tells it to cool the rooms to 22C when people are present or likely to be present, but not to cool it when people are absent or likely to be absent an agent? I think not.
Is a troubleshooting guide with lots of if-then-else branch points an agent? No.
Consider a tool that I write which will write a program to solve a problem I am interested in solving. Let say I want to build a medical robot which will wander the world dispensing medical care to anyone who seeks its help. The code I write to implement this has a lot of recursion in the sense that my code looks at symptoms of the current patient and writes treatment code based on its database and the symptoms it sees, and modified the treatment code based on reactions of the patient.
As long as this robot continues to treat humans medically, it does not seem at all agenty to me. If it started to develop nutrition programs and clean water programs, it would seem somewhat agenty to me. Until it switched professions, decided to be a poet or a hermit or a barista, I would not think of it as an agent.
As long as my tool is doing what I designed it to do, I don't think it shows any signs of wanting anything.
What if your robot searched the medical literature for improved treatments? What if it improved its ability to find treatments?
In the spirit of "satisficers want to become maximisers" here is a somewhat weaker argument (growing out of a discussion with Daniel Dewey) that "tool AIs" would want to become agent AIs.
The argument is simple. Assume the tool AI is given the task of finding the best plan for achieving some goal. The plan must be realistic and remain within the resources of the AI's controller - energy, money, social power, etc. The best plans are the ones that use these resources in the most effective and economic way to achieve the goal.
And the AI's controller has one special type of resource, uniquely effective at what it does. Namely, the AI itself. It is smart, potentially powerful, and could self-improve and pull all the usual AI tricks. So the best plan a tool AI could come up with, for almost any goal, is "turn me into an agent AI with that goal." The smarter the AI, the better this plan is. Of course, the plan need not read literally like that - it could simply be a complicated plan that, as a side-effect, turns the tool AI into an agent. Or copy the AI's software into a agent design. Or it might just arrange things so that we always end up following the tool AIs advice and consult it often, which is an indirect way of making it into an agent. Depending on how we've programmed the tool AI's preferences, it might be motivated to mislead us about this aspect of its plan, concealing the secret goal of unleashing itself as an agent.
In any case, it does us good to realise that "make me into an agent" is what a tool AI would consider the best possible plan for many goals. So without a hint of agency, it's motivated to make us make it into a agent.