I would only count debts toward the specific peoples directly affected; e.g. the Spanish Empire lived off Bolivian silver, the Belgians worked the Congolese to death, and the United States is literally built on stolen Native land. Those examples and many others allow for a case in favor of reparations.

However, the passage of time sometimes blurs the effects of exploitation and aggression. Should the UK sue Denmark for the Norman Conquest? Should Italy sue Germany because Germanic tribes destroyed the Roman Empire? Should Hungary sue Mongolia for what the Golden Horde did to them? I admit I don't know how to answer to that in a way that is consistent with my first paragraph.