I say "Jesus Christ!" when I'm startled. I say "Thank God" when I'm relieved, and "Oh, God..." when I'm hung over. I say "God damn it!" when I'm frustrated. But all of these feel like idiomatic expressions of emotion to me, not like invocations of divinity. And I'm not too worried about this; signaling atheism isn't much of a priority for me, and there's linguistic precedent for it, by Jove.

If you're seriously worried about expressing yourself this way, I might suggest implementing a God jar, with the proceeds going to the atheist nonprofit of your choice.