You may try thinking about how you would feel if you had a button that collapsed a mine in Chile if you pushed it. Would you push it on a whim just because miners dying in Chile doesn't necessarily make you sad or would you suddenly feel a personal connection to those miners by means of the button you had to control their fate?

I wouldn't push the button because

1. fear that my action might be discovered,
2. feeling guilty of murder,
3. other people's suffering (the miners' when they would be dying and their relatives' afterwards) having negative utility to me,
4. "on a whim" doesn't sound as reasonable motivation,
5. fear that by doing so I would become accustomed to killing.

If the button painlessly killed people without relatives or friends and I were very certain that my pushing would remain undiscovered and there were some minimal reward for that, that would solve 1, 3 and 4. It's more difficult to imagine what would placate my inner deontologist who cares about 2; I don't want to stipulate memory erasing since I have no idea how I would feel after having my memory erased.

Nevertheless if the button created new miners from scratch, I wouldn't push it if there was some associated cost, no matter how low. Assuming that I had no interest in Chilean mining industry.