It took a bit of an outsider-- a mediocre swimmer who was nuts about the sport-- to apply physics to swimming, and he didn't start by thinking about physics, he started by noticing that naturally good swimmers didn't look the same in the water as naturally bad swimmers.

There may still be plenty of low-hanging fruit left to apply physics to sports. Or maybe not-- that site mentions that, because of the high drag from water, efficiency is more crucial in swimming than in other sports.