I'm not saying Paleogenes are not a possible explanation, but I haven't seen much in terms of such evidence like:

• I don't think it's so common for people to go around the world, and actually verify that people in statistically significant number of virtually isolated tribes do have preferences for larger breasts etc. So universality is more postulated than actually empirically found. Even if you find extremely common behaviour, it can still be memetic, as a lot of memes are copied from parents to children. How many behaviours are empirically known to be universal?
• Mutations to such genes would be non-lethal, and some would only mildly reduce inclusive fitness. We have plenty of genetic diseases in the population affecting more important genes. So how come we haven't discovered mutations in the wild that alter genes controlling this supposedly genetic behaviour.
• We know very few genes obviously linked with behaviour, and saying it's coded by emergent interaction of multiple genes is just handwaving away the problem. There's a pretty low ceiling of how much can be straightforwardly coded this way, and I'd expect some serious evidence of some mechanism of complex genetically coded behaviour.