I think there's a distinction. I have the right to vote. If I were black, living in particular areas or time periods, I might still have to worry about whether that right will end up mattering in the real world. I guess I'd say that "privilege" is the word for rights that are not fairly implemented in practice.

(though I don't actually believe in 'rights' as something that exists in the abstract. They're conventions we use because people collectively prefer to have them.)