This is often a good idea in mathematics. Two concepts that are equivalent in some context may no longer be equivalent once you move to a more general context; for example, familiar equivalent definitions are often no longer equivalent if you start dropping axioms from set theory or logic (e.g. the axiom of choice or excluded middle).
Outside of mathematical logic, some familiar examples include:
Here's another installment of rationality quotes. The usual rules apply: