That seems like a magical sort of time travel; the sort that is conceivable but unphysical; a garbage in, garbage out type-deal. I'd echo Viliam's remark on no absolute reference frames. I think it helps to imagine how you might actually, physically perform time travel. I usually see it suggested that one create a wormhole and accelerate one of the mouths. Time dilation will cause an observer in the reference frame of the accelerated mouth to experience less subjective time. Travel through the stationary mouth, and you'll apparently come out of the accelerated mouth at an earlier moment in time. No magic involved.