the source of the fear can't be entirely evolutionary - I could be not the least bit afraid of the dark in normal life but become very afraid when in a supposedly haunted area.

The human brain has evolved several mechanisms that are useful for detecting other agents, but which sometimes produce false-positives. For example, we have some sort of hardware specifically dedicated to recognizing faces (presumably detect the pattern of two eyes, a nose and a mouth). There exist people who have damage to their brain such that they seem mentally perfectly normal, except that they have trouble detecting people in photographs, for example.

When this hardware produces a false positive, then at some (subconcious?) level, you are detecting that there is someone here. But when you use your conscious mind to scan your environment, you don't see anyone. This is what causes unease, and the feeling of "hauntedness".

I believe there's also a part of the brain which has evolved to detect agency (perhaps to help with dealing with "Theory of Mind" and emulating other people's minds for social purposes?) and "false positives" in detecting agency may be one explanation of religion.