One reason to be skeptical of labeling objects as colored is that different animals have 'shifted' spectrums. For example, many flowers that look uniformly colored to us will appear to be two different colors to bees. So is the flower one color or two? Do humans get full domain over seeing colors and animals are relegated to being 'wrong'? Why? In fact, most daytime birds have four cones, as opposed to our three. They can make many, many more color discriminations than we can. So we're 'colorblind' relative to the birds.