There are systems (relevant logic, for example) which do not collapse under one contradiction - to some extent, the fragility of classical logic is due to very strong assumptions that were built into it about how powerful math will turn out to be before Godel's incompleteness and other undecidability results were discovered (and, at least in pop math like you and I are familiar with, they're still not really fully digested).

Charlie Stross has also written post-arithmetic-consistency sf: "Dark Integers". I'm a little surprised that Ted Chiang's story didn't contain any attempts to build devices to exploit the inconsistency.