There is nothing offensive about you having high karma. It is offensive that you you abused a system that a lot of us rely on for evaluating content and encouraging norms that lead to the truth. Truth-seeking is a communal activity and undermining the system that a community uses to find the truth is something we should punish. It's similar to learning that you had lied in a comment.

I imagine the vast majority of your karma is not ill-gotten, I have no problem with you having it.

Anyway, I haven't voted you down for precedent setting reasons.