Hailing from secular Britain I wasn't aware of the distinction. Affirmation actually sounds more religious to me. I'd never particularly associated the idea of an oath with religion but I can see how such an association could sour one on the word 'oath'.