I feel like the asymmetry in experience between red-haired and non-red-haired is important here.
Like, if you don't have red hair, joking about being hostile towards red haired people feels entirely innocuous. Even mild assault isn't that big of a deal (imagine playfully kicking someone in the shins). You're just joking, right?
If you do have red hair being told that you're subhuman or being assaulted because of your hair color doesn't feel nearly as inconsequential.
Spreading the meme feels harmless to the meme-spreaders, but the meme itself feels harmful to the victims.
This is what is commonly meant by the term "privilege" these days: an advantage you have whether you know you do or not, and particularly if you don't know.
Edit: as already noted by JulianMorrison.
There's a tradition in England - I don't know how old - of abusing red-headed people. It's a genuine prejudice in England. From this facebook page:
This spread to the US in 2005, when Cartman tried to incite violence against redheads in a South Park episode with "Kick a Ginger Day".
What's interesting is how this meme is spreading in the US: As humor. This meme is promoted by sites like CollegeHumor.com and MyLifeIsAverage.com, which mine it as a source of ironic humor. The Cheezburger Network is pushing ginger-hatred almost as aggressively as they push pedophilia as a fount of humor.
Are humans capable of, collectively, keeping real and humorous/ironic racism separate? No, they are not. What South Park "kicked" off as an ironic commentary on racism is becoming actual racism.
One clue that you're going too far in your ironic humor is when you start finding the real thing funny.
Do humans have an instinctive need to bond over shared prejudices? Is combating racism a game of whack-a-mole, in which society invents new prejudices to replace the ones being taken away?