Insufficient opportunity and brainwashing.

I think this is basically correct, but I'd express it in terms of cultural inertia rather than brainwashing. It's not (usually) part of a planned campaign of retention, it's just that learning a completely different culture and language and set of survival skills is a huge risk and would take a huge amount of effort: it might be attractive in marginal cases, but most people would likely feel they had too much to lose. Particularly if the relationship between the cultures is already adversarial.