I don't perceive this as a problem unique to design. It's also a problem in education, business consulting, medicine, etc. Any time consumers don't have an easily available means of assessing product quality, there is little incentive for businesses to maximize effectiveness, so there will often be a huge gap between best practices and actual practices.
So I'm working for a friend's company at the moment (friend is a small business owner who designs websites and a bit of an entrepreneur) anyway, I've persuaded him that we should research the empirical literature on what makes websites effective (which we've done a lot of now) and to advertise ourselves as being special by reason of doing this (which we're only just starting to do).
One thing that I found absolutely remarkable is how unfilled this space tends to be. Like a lot of things in the broad area of empirical aesthetics it seems like there are a lot of potentially useful results (c.f.http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3485842/ ), but they're simply not being applied- either as points of real practice or of marketing differentiation.
A fascinating gap.