There's a defeasible presumption that if I project matter and/or energy into you, and that directly causes you to suffer, I'm in the moral wrong.

Nope. Suffering is not a blank check to control a shared environment. And you recognize that as such later on:

we allow people to inflict certain levels of noise, pollution, etc., on each other.

But this doesn't mean that everything allowed under the law - even an optimal law - is morally OK.

Yes. If you're not a theocrat, legal and moral are not the same things.