I used the "step" language because people on the Internet are depressingly literal, and if I just called them a utility monster, someone would tell me that since they're clearly not demanding I spend infinite resources on them to increase their utility, they're not really a utility monster.
If everyone in society gets to do that, even with a few free-riders, everyone ends up better off.
No, they don't, because conceding to such demands affects motivations. With the number of free riders we have now, giving in to everyone's demand isn't going to cause too much damage from giving in to the free-riders. But that ignores the role of giving in towards making more free-riders.
Also, remember that we're talking about someone asking you to turn your noise down even though your volume is already within what is allowed by regulations or social norms.
My hidden secret goal is to understand the sentiments behind social justice better, however I will refrain from asking questions that directly relate to it, as they can be mind-killers, instead, I have constructed an entirely apolitical, and probably safe thought experiment involving a common everyday problem that shouldn't be incisive.
Alice is living in an apartment, she is listening to music. The volume of her music is well within what is allowed by the regulations or social norms. Yet the neighbor is still complaining and wants her to turn it down, claiming that she (the neighbor) is unusually sensitive to noise due to some kind of ear or mental condition.
Bob, Alice's friend is also present, and he makes a case that while she can turn it down basically out of niceness or neighborliness, this level of kindness is going far beyond the requirements of duty, and should be considered a favor, because she has no ethical duty to turn it down, for the following reasons.
1) Her volume level of music is usual, it is the sensitivity level of the neighbor that is unusual, and we are under no duty to cater to every special need of others.
2) In other words, it is okay to cause suffering to others as long as it is a usual, common, accepted thing to do that would not cause suffering to a typical person.
The reasons for this are
A) It would be too hard to do otherwise, to cater to every special need, in this case it is easy, but not in all cases, so this is no general principle.
B/1) It would not help the other person much, if the other person is unusually sensitive, the problem would not be fixed by one person catering to them. A hundred people should cater to it, after all there are many sources of noise in the neighborhood.
B/2) In other words, if you are unusually rude, reducing it to usual levels of rudeness is efficient, because by that one move you made a lot of people content. But if you are already on the usual levels of rudeness and an unusually sensitive person is still suffering, further reduction is less efficient because you are only one of the many sources of their suffering. And these people are few anyway.
C) Special needs are easy to fake.
D) People should really work on toughening up and growing a thicker skin, it is actually possible.
Polls in comments below
Please explain your view in the comments.