Perhaps sparklines would work for this. They compress the recent history of a measurement in a space-efficient way which can fit inline with text.
They would fit neatly next to the upvote / downvote buttons.
However nicely they would fit, they should not be used, though. I am in the mind of the diagram about the effect Google's proxy is having on web content - to the extent that karma is not a perfect proxy for good content, sparklines will make it easier to identify that proxy and where it can be gamed.
People who go back and downvote every post or comment a Less Wrong user has ever made, please, stop doing that. It's a clever way to pull information cascades in your direction but it is clearly an abuse of the content filtering system. It's also highly dishonorable. If you truly must use such tactics then downvoting a few of your enemy's top level posts is much less evil; your enemy loses the karma and takes the hint without your severely biasing the public perception of Less Wrong's discourse.
(I just lost over 200 karma in a few minutes and that'll probably continue for awhile. This happens to me every few weeks. Edit: I mean it's been happening every few weeks for a few months for a total of only three or four. Between 400 and 700 karma lost total I think? I don't mean to overstate the problem.)