Asset allocation matters too. Vanguard target retirement funds follow the conventional wisdom (more stocks when you're young, more bonds when you're older) and are pretty cheap. Plowing all new investments into a single target-date fund is good advice for most people*.

I implemented a scheme to lower my expenses from 0.17% to 0.09%, but it was not worth the time, hassle, and tax complications.

*People who should do something more complicated include retirees, who should strongly consider buying an annuity, and people who are saving to donate to charity.