That's because it's easy to misvalue assets if you're disconnected from the production process. So when you have specialized bookkeepers, others will typcially see them as ignorant of the true value of the assets, and associate this with bookkeeping per se, rather than bookkeeping with a screwy incentive structure and/or knowledge flows. Because this is the context in which most people interface with accountants, they tend to be associated with misvaluing assets. And thus:

"Beancounters didn't think a soldier's life was worth 300 [thousand dollars]." -- Batman Begins

Edit: Sorry, I forgot to translate all that: P(observe "accountant" | believe accountant misvalued assets) > P(observe "accountant" | ~believe accountant misvalued assets)