Many neurotypicals I have spoken to will take really extreme positions on the fat man trolley problem, saying that they wouldn't push the fat man off the bridge even if a million people were on the trolley.

Eh, as I've argued before on LW, there are utilitarian, AS-compatible justifications for such a position: specifically, that your heroic act shuffles around the risk profiles of various activities in unpredictable ways, thus limiting the ability of people to manage risks, leading them to waste significant resources (perhaps exceeding the amount that would otherwise save more than a million lives) returning to their preferred risk profile.

The key part:

By intervening to push someone onto the track, you suddenly and unpredictably shift around the causal structure associated with danger in the world, on top of saving a few lives. Now, people have to worry about more heros drafting sacrificial lambs "like that one guy did a few months ago" and have to go to greater lengths to get the same level of risk.

In other words, all the "prediction difficulty" costs associated with randomly changing the "rules of the game" apply. Just as it's costly to make people keep updating their knowledge of what's okay and what isn't, it's costly to make people update their knowledge of what's risky and what isn't (and to less efficient regimes, no less).

Note that this doesn't argue for a deontological prohibition, but rather, argues about the consequences of sudden deviations from social norms, without assumption of their categorical justness.

ETA: In terms of Timeless Decision Theory, you could put it this way: if people knew that bridge-walkers are drafted for deadly work on a moment's notice, it's much less likely you'd have a fat person handy to begin with. So, the way TDT calculates probabilities, the EU of pushing the fat guy off is very small on account of its low TDT-probability, eliminating the supposed utility gain.