I found an article that suggests that removing mutational load would significantly improve traits, especially intelligence. Mutational load is all the random mutations you carry, many of which cause slight detrimental effects.

I don't think this would be anywhere near as controversial, and wouldn't decrease the existing genetic variation.