It doesn't offend me when other people say it. It's true that they aren't thinking -- it's just a social pleasantry, people don't often think about those! -- but it's meant in a nice way. Also, perhaps this is a cultural difference (I'm in the relatively irreligious UK), but I think most people who say it here don't mean it as a religious invocation in the slightest. I just don't particularly like expressing that connotation myself.