If your definition of complexity says noise is complexity, then you need a new definition of complexity.

Yes, many useful definitions, like entropy measures or Kolmogorov complexity, say noise is complexity. But people studying complexity recognize that this is a problem. They are aware that the phenomenon they're trying to get at when they say "complexity" is something different.