The user most likely to engage in retributive downvoting are those who engage in hostile debate and subsequently have low karma ratio's themselves (VoiceofRa has 68% favourability). Perhaps you could disable downvoting functionality for those with a karma ratio lower than 80%? Considering that poor quality of contributions is another big factor for low karma ratios this measure would have the added benefit that our most competent users have more power.