Agreed - we do use such labels for many aspects of sounds, but with the exception of labelling musical notes, sound-related terms tend to refer to multidimensional characteristics of sounds, not just frequencies (although in many cases, related to frequency). This leads to a situation in which sound quality terms may tend to be less discrete than basic colour words; if so one would predict less categorical perception than is observed for basic colour words.

I think we can also see this same phenomenon in colour as well - once we go beyond a language's basic colour terms where colour labels are not nearly so contrastive (mauve/fuschia?).