From the article linked (synesthete is a keyword explained in the article):

They found a synesthete who was color blind. That may seem strange, but what it really means is that the subject had problems with his retina that left him able to distinguish only an extremely narrow range of wavelengths when looking at most images in the world — his brain was fine, but his eyes weren’t quite up to the job. But when he saw certain numbers, he experienced colors that he otherwise never saw.

And that, I guess, answers that question. Awesome.