first: fairness is not the same as morality*. (ignore this point if you think fairness is a crucial thing to measure in morality)

Second: Most people seem to be mutually primary. You're getting priority from someone and giving it to them in return, but you can also have others. It's rare that someone is poly amorous but demands monogamy from their loves. Even if they did, this leads into

Third: We're talking about consensual relationships here. If you want priority, then you can date only people who will give you priority. Hell, if you want to date someone and have them give you priority, and NOT give them priority in return, as long as they agree why should this be a problem?

fourth: You seem to be viewing this as unfairly advantageous to a poly person, because they get "priority" and also bonus sex, but it's also advantageous to all the secondaries, who presumably don't care about or at least don't need priority, and would have less romance without the poly person.

*to elaborate: People have different preferences, often vastly different. Unless you take this into account, naive views of fairness lead to perverse results. Imagine two people: Tom hates cake and loves pie, and Dave hates pie and loves cake. They live in an unfair universe where they have 3 cakes and 1 pie to divide between themselves. It seems "Unfair" to give Dave 3 cakes and give Tom 1 pie, yet this is the best outcome. An extra cake won't do tom any good, and so dave is being deprived for no reason. Now replace desire for cake with desire for sex and imagine a person who has lot of desire for sex but is in love with someone with none. Why deprive both of them of love? Add another person, and every individual involved is happier.