then the continuous nature of traits like height shouldn't be much of a barrier to being concerned about it.

Indeed. Parts of the internet divide people (at least men) by height with words like "manlet", and specific numbers for the cutoff.