Yes, it would be reasonable to have separate per-user/user link limits. Though the limits could be a function of the number of comments that user made recently -- if someone gets into a manic mode and posts dozens of trash comments in a few hours...
Right now, users with sufficiently high karma have access to the vote buttons on the userpages of users with sufficiently low upvote percentage, as far as I can tell to enable this sort of downvoting. It seems likely that exemptions could be baked it to these rate limits just as easily.
I've gotten sufficient evidence from support that voiceofra has been doing retributive downvoting. I've banned them without prior notice because I'm not giving them more chances to downvote.
I'm thinking of something like not letting anyone give more than 5 downvotes/week for content which is more than a month old. The numbers and the time period are tentative-- this isn't my ideal rule. This is probably technically possible. However, my impression is that highly specific rules like that are an invitation to gaming the rules.
I would rather just make spiteful down-voting impossible (or maybe make it expensive) rather than trying to find out who's doing it. Admittedly, putting up barriers to downvoting for past comments doesn't solve the problem of people who down-vote everything, but at least people who downvote current material are easier to notice.
Any thoughts about technical solutions to excessive down-voting of past material?