Minkowski spacetime is primarily concerned with causal distance; whether event A can be causally related to event B. Time has a negative sign when you're considering causality, because your primary goal is to see whether any effect from event A could have been involved in event B. Using the Minkowski definition of time, an object A ten million light years away from object B has a negligible spacetime distance from that object ten million years in the future and ten million years in the past from any given point in time.