Peter Richerson here says:

The hypothesis is that as the avenues of nonparental transmission increase, cultural variants that exploit us to reproduce themselves at the expense of our genetic fitness have had an ever easier time spreading.

He goes on to give some data about that.

My favourite example of this sort of thing is the demographic transition in meme-rich Japan. The native humans there live for a long time - but they are practically sterile.