I would taboo and translate that use to yield something like "To make sense of rationality in an agent, one needs to accept/assume/stipulate that the agent sometimes acts with a purpose in mind. We need to understand 'purpose', in that sense, to understand rationality."

Thanks, yes. This is very clear. I can buy this.

But I think I understand this kind of purpose, identifying it as the cognitive version of something like "being instrumental to survival and reproduction". That is, it is possible for an outside observer to point to behaviors or features of a virus that are instrumental to viral survival and reproduction.

Sorry if I'm slow to be getting it, but my understanding of your view is that the sort of purpose that a bacterium has, on the one hand, and the purpose required to be a candidate for rationality, on the other, are, so to speak, different in degree but not in kind. They're the same thing, just orders of magnitude more sophisticated in the latter case (involving cognitive systems). This is the idea I want to oppose. I have tried to suggest that bacterial purposes are 'merely' teleonomic -to borrow the useful term suggested by timtyler- but that human purposes must be of a different order.

Here's one more crack at trying to motivate this, using very evidently non-scientific terms. On the one hand, I submit that you cannot make sense of a thing (human, animal, AI, whatever) as rational unless there is something that it cares about. Unless that is, there is something which matters or is important to it (this something can be as simple as survival or reproduction). You may not like to see a respectable concept like rationality consorting with such waffly notions, but there you have it. Please object to this if you think it's false.

On the other hand, nothing in nature implies that anything matters (etc) to a thing. You can show me all of the behavioural/cognitive correlates of X's mattering to a thing, or of a thing's caring about X, and provide me detailed evolutionary explanations of the behavioural correlates' presence, but these correlates simply do not add up to the thing's actually caring about X. X's being important to a thing, X's mattering, is more than a question of mere behaviour or computation. Again, if this seems false, please say.

If both hands seem false, I'd be interested to hear that, too.

At the level of a bacterium, there are second-messenger chemicals that symbolize or represent situations that are instrumental to survival and reproduction. At the level of the nematode, there are neuron firings serving as symbols. At the level of a human the symbols can be vocalizations: "I'm horny; how about you?". I don't see anything transcendently new at any stage in this progression, nor in the developmental progression that I offered as a substitute.

As soon as we start to talk about symbols and representation, I'm concerned that a whole new set of very thorny issues get introduced. I will shy away from these.

Let me try putting that in different words: "Norms are in the eye of the beholder. Natural science tries to be objective - to avoid observer effects. But that is not possible when studying rationality. It requires a different, non-reductionist and observer dependent way of looking at the subject matter." If that is what you are saying, I may come close to agreeing with you. But somehow, I don't think that is what you are saying.

"It requires a different, non-reductionist ... way of looking at the subject matter." -I can agree with you completely on this. (I do want however to resist the subjective, "observer dependent" part )