You fail to understand what rich and poor mean. While a rich person may be using dollars to keep score, a poor person is using them to stay alive.

Do you really think that someone with a million dollars could care about each one of them as much as someone who has only one dollar cares about his one dollar? That the million dollar owner could be more devastated by the loss of two of his million dollars than the one-dollar owner will be by the prospect of not eating if he loses his one dollar?