While some people tried to appeal to non-linear aggregation, you would have to appeal to a non-linear aggregation which was non-linear enough to reduce 3^^^3 to a small constant. In other words it has to be effectively flat. And I doubt they would have said anything different if I'd said 3^^^^3.

And why should they consider 3^^^^3 differently, if their function asymptotically approaches a limit? Besides, human utility function would take the whole, and then perhaps consider duplicates, uniqueness (you don't want your prehistoric tribe to lose the last man who knows how to make a stone axe), and so on, rather than evaluate one by one and then sum.

Scope insensitivity - extremely sublinear aggregation by individuals considering bad events happening to many people - can lead to mass defection in a multiplayer prisoner's dilemma even by altruists who would normally cooperate. Suppose I can go skydiving today but this causes the world to get warmer by 0.000001 degree Celsius. This poses very little annoyance to any individual, and my utility function aggregates sublinearly over individuals, so I conclude that it's best to go skydiving. Then a billion people go skydiving and we all catch on fire. Which exact person in the chain should first refuse?

The false allure of oversimplified morality is in ease of inventing hypothetical examples where it works great.

One could, of course, posit a colder planet. Most of the population would prefer that planet to be warmer, but if the temperature rise exceeds 5 Celsius, the gas hydrates melt, and everyone dies. And they all have to decide at one day. Or one could posit a planet Linearium populated entirely by people that really love skydiving, who would want to skydive everyday but that would raise the global temperature by 100 Celsius, and they'd rather be alive than skydive every day and boil to death. They opt to skydive at their birthdays at the expense of 0.3 degree global temperature rise, which each one of them finds to be an acceptable price to pay for getting to skydive at your birthday.