I don't think that's the reason, because if it was it would apply regardless of which words you use, whatever their literal meaning, so long as it's reasonably unambiguous in the context (why would “the ladies' room” or “talk to a man about a horse” be any less problematic, when the listener knows what you mean?)

When a woman goes to the ladies room she might also go to fix up her makeup or hairstyle. Secondly words matter. Words trigger thoughts. If you speak in deep metaphars you will produce less images than if you describe something in detail.

(ISTM that “toilet” is less often replaced by euphemisms in BrE than in AmE).

Americans are more uptight about intimicy, so that fits nicely. They have a stronger ban on cureswords on US television than in Great Britian. I would also expect more people in Bible Belt stats to use suuch euphemisms than in California.