For example, I'd feel less masculine wearing a pink shirt around North America, but guys in China did so fairly commonly, and I'd expect to find considerable variation in this across time and cultures, so I consider it a bad idea, or at least pointless, for color based gender norms to be overtly encouraged.

You'd be right; the association of pink with femininity is fairly recent.

It's not clear to me that putting a lot of effort into eliminating overt caste markers is the best way to go if you're interested in weakening caste, though.