I have been vaguely thinking of writing a top level article about psychological essentialism for a while, but this seems like a good place to just point to it. There is a large body of research on the subject and the best summary of it I know is Susan Gelman's "Essentialism in Everyday Thought".

The process of imputing causation to visible characteristics from an unseen inner variable seems to be something humans subconsciously and automatically do in some situations. You can do studies of essentialist reasoning in children by asking "Suppose an X was raised by Ys, in case C would the X tend to do this X-ish things or this Y-ish things?" Very young children will give different answers for difference values of C, and in some cases (like with human language) they are clearly wrong which makes it seem like maybe they are executing a domain specific "complex essentialist cognitive module" rather than reasoning from observed evidence.

If you want to see some back and forth on the subject instead of just taking one author's word for it, here is a 1993 criticism of the essentialist research program by Susan Jones and Linda Smith and a rebuttal by Susan Gelman.

My impression is that essentialist cognition is enormously common because it is enormously useful. There is an essence for ducks and its called the duck genome. Hypothesized essences have been mapped to one or more physical mechanisms so often that mostly I just assume that this can be done for all essences that are coherent. The mechanisms generally have approximately the pragmatic properties normally ascribed to essences (like immutability) but it doesn't match human intuitions perfectly because, whether its a democracy or an ecology or a painting "it's just an arrangement of atoms, right?"

After I internalized this view of the world, I gained a measure of sympathy for people who believed in supernatural theories because those situations became intelligible to me as people having working models for reality using the built in tools of human reasoning. "Evil ghosts in the swamp" model "malaria" well enough to explain why to avoid swamps while looking for food. "God" models "inscrutable optimization processes stronger than me" enough to help people make a kind of peace with the fact of limited power and explain the appearance of order in the world while looking for food.

When I need to "argue past the essence" without giving the whole cogsci backstory I try to point out the "magic sparks" buried in the theory and then denaturalize the sparks by hypothesizing vaguely realistic complex mechanisms in their place. Like sometimes people deploy the concept of IQ as working like a magic spark that is more or less bright, and you can denaturalize it by breaking it into pieces that can obviously vary independently like working memory capacity, vocabulary acquisition, "good nutrition, good sleep, and the time to cultivate your mind", and so on.

But most of the time, a theory with a spark is perfectly good for making the predictions you need to avoid scary situations and find food. I would expect an AGI worth its salt to have some internal mechanism that functions roughly to "notice the sparks" and use them as appropriate and push deeper as dictated by opportunity and need :-)