It is entirely in the eye of the beholder, it's a subjective opinion and as such there is no "actually" about it.

As mentioned in the original post, the same can be said of race: I may subjectively prefer white people.

You might bite the bullet here and say that yes, in fact, racism, sexism etc. is morally acceptable, but I think most people would agree that these __isms are wrong, and so speciesism must also be wrong.