Arriving late is usually accompanied with others waiting for me and not being sure when I manage to arrive, which is very annoying. With early leaving this effect is absent. If I let the others know in advance that I will arrive late, the effect would disappear as well, i.e. they will be no more angry at me than they would if I left too early.