This is true, but apologists have done quite a lot of work trying to reconcile the claims of their religion with the existing physical record, and I don't think their efforts are inferior to those of more mainstream Christian apologists.

Biblical literalism entails the belief that God wiped out the human race minus one family with a world-encompassing flood which subjected all terrestrial animals to a population bottleneck of one to seven mating pairs per species, which would have to have spread out from one geographic location to repopulate the globe. Compared to that burden of improbability, the empirical claims of Mormonism are a paltry addition, and there are no shortage of apologists to defend that claim.