I think the reason that it's considered funny is that, while many consider racists stupid, it is almost unbelievably silly to think of someone as sub-human because of their hair color. It is true that I haven't actually experienced prejudice of this kind but many of my red-haired friends routinely make ginger related jokes. At least in my circles, racism against red-haired people is entirely ironic.
It might be worth noting that we were first exposed to anti-ginger feelings though humor, not from any legitimately racist source.