There's also the observed matter-antimatter asymmetry. Even if you want to argue that virtual electrons aren't real and thus don't count, it still seems to be the case that there are a lot more electrons than positrons. If it was just one electron going back and forth in time, we'd expect at most one extra electron.

Not to mention the fact that positrons = electrons going backwards in time only works if you ignore gravity.