I don't do so unless I feel very strongly about something - a comment explaining my disagreement or disapproval is more productive
Sometimes, sometimes not. If I see a comment that's most likely wrong but sounds belligerent, confused or pedantic (or was posted by someone with a track record of being belligerent, confused or pedantic) then I sometimes predict that posting a reply would just suck me into a futile, time-wasting argument. In those cases I happily downvote without bothering to reply. (Other times I prefer to reply to and downvote very misleading comments.)
People who go back and downvote every post or comment a Less Wrong user has ever made, please, stop doing that. It's a clever way to pull information cascades in your direction but it is clearly an abuse of the content filtering system. It's also highly dishonorable. If you truly must use such tactics then downvoting a few of your enemy's top level posts is much less evil; your enemy loses the karma and takes the hint without your severely biasing the public perception of Less Wrong's discourse.
(I just lost over 200 karma in a few minutes and that'll probably continue for awhile. This happens to me every few weeks. Edit: I mean it's been happening every few weeks for a few months for a total of only three or four. Between 400 and 700 karma lost total I think? I don't mean to overstate the problem.)