Indeed... that is a bit of a weak point of evolutionary psychology; it's likely that quite some behaviours have an evolutionary background, but it's very hard to proof -- and in quite some cases, you could 'proof' opposite conclusions; as Chomsky said:

"You find that people cooperate, you say, ‘Yeah, that contributes to their genes’ perpetuating.’ You find that they fight, you say, ‘Sure, that’s obvious, because it means that their genes perpetuate and not somebody else’s. In fact, just about anything you find, you can make up some story for it."

This is not to say that EP is wrong, just that things are hard to proof. E.g., compared to biological evolution, there are no fossils of ancient behaviours or intermediate forms and so on.