Reinforcement learning isn't trivial, but it might be kind of modular. Evolution doesn't have to create reinforcement learning from scratch every time, it can re-use the existing structures and just hook in new inputs.

Learning takes a lot of overhead. Much less overhead if you can simply be born knowing what you need to know, instinctively. Konrad Lorenz believed that instincts somehow develop out of learned behavior. He looked at related species where one had an instinctual behavior and the other had to learn the same behavior. It takes so long to learn instincts by mutation and selection, it's intuitively obvious that learning should come first and then be replaced by instinct over time. But Lorenz didn't propose a mechanism for that to happen and I don't know a mechanism either.

We do so much less instinct than other animals that I'm tempted to think we haven't spent much time in consistent predictable environments. Maybe we've always lived by disrupting ecosystems and surviving in the chaos -- a role we would manage better than animals that do less flexible thinking.