I sort of see what you're saying, but are you sure with that change, you still want to respond "yes" to that question? My problem with responding with "yes" to something like that is that the person will end up criticizing me based one what he's read of other people (who also call themselves by that label). The most I would ever do is say something like, "Yeah I find useful a lot of the stuff said in paleo circles." That way they have an idea of where I stand, but they'll be less likely to assume prematurely that I believe something I don't.

Also, if someone assumes I'm some label, such as an "utilitarian", if it does fit to some extent, I'll say something like, "Well, I wouldn't say that, but I guess some of my views are similar to other people who have called themselves that." It does wonders for my ability to avoid people assuming I hold beliefs I don't because of their interpretation of what other people who used that word thought. Also it makes it feel less like I have to defend other people in order to defend myself. I choose very selectively on what or who I endorse, and I phrase it carefully.

I'd recommend perhaps changing your tactic to answering that question with something like, "Well I do agree with a lot of what the feminist community says, especially what's written by [name of whoever you think most represents your views]." That would be much safer, in my opinion. By this technique you may be able to get even better at updating than you already have with the first shift. Just a thought though.

(Another problem being that many labels or terms have tons of meanings, for example the term "utilitarian" referring to all sorts of different things; I'm sure the label "feminist" isn't much different.)