Of course, this has its own moral dilemmas as well - such as the fact that you're as good as dead for your loved ones in the timeline that you just left - but generally smaller than erasing a universe entirely.

You could get around this by forking the time traveler with the universe: in the source universe it would simply appear that the attempted time travel didn't work.

That would create a new problem, though: you'd never see anyone leave a timeline but every attempt would result in the creation of a new one with a copy of the traveler added at the destination time. A persistent traveler could generate any number of timelines differing only by the number of failed time travel attempts made before the succesful one.