In some fields of math a lot of literature isn't published in English so, knowing other languages will help you. On the other hand I still don't think that there's memory interference.

If you learn a new English math term ideally you might want to wait a week or two till you learn the French, Russian or German term for the new concept. But two terms won't be that big of a problem even if you don't wait that week.