This would mean that all our decisions were dominated by tiny-seeming probabilities (on the order of 2-100 and less) of scenarios where our lightest action affected 3↑↑4 people... which would in turn be dominated by even more remote probabilities of affecting 3↑↑5 people...

I'm pretty ignorant of quantum mechanics, but I gather there was a similar problem, in that the probability function for some path appeared to be dominated by an infinite number of infinitessimally-unlikely paths, and Feynman solved the problem by showing that those paths cancelled each other out.