I'd claim that there is a distinct concept of "purpose" that people use that doesn't entail an agent with that purpose. It may be a pretty unhelpful concept, but it's one that people use. It may also have arisen as a result of people mixing up the more sound concept of purpose.

I think you're underestimating people who worry about "ultimate purpose". You say they "don't even understand the context", as opposed to people who "understand the full context of the concept". I'm not sure whether you're just being a linguistic prescriptivist here, but if there are a whole bunch of people using a word in a different way to the way it's normally used, then I'm inclined to think that the best way to understand that is that they mean something different than that, not that they're idiots who don't understand the word properly.