The label can still have political use even if it doesn't have practical use.

For an example, let's go with Wiggins, people with green eyes and black hair. Wiggins are untrustworthy, and put too much ketchup on their fries, everyone knows that. A minor political party could even sprout up in Australia on a Wiggin-centric platform. But then some statisticians raise the point that we don't have strong evidence differentiating Wiggins from other people. What does the political party in Australia do? "If you're not with us, you're with the Wiggins! How can they say there's no difference between you and a Wiggin, when we can so clearly see the difference? Remember to vote to protect Australia from the Wiggins!"

Sure, at some point reality will become inconvenient. But it takes more than mere evidential neutrality to stop the Australian Anti-Wiggin Party - all it means is people have to use their intuitions.