You have special hardware for simulating others' cognition. Neurologically, imagining how someone feels is a completely different thing from imagining a collection of 35 apples.

I can't tell what context you're getting this from, but I've seen "You don't understand how I feel!" used as bad epistemology.

My sister's a heroin addict, and she'll use the fact that I've never been addicted to heroin or experienced opioid withdrawals as a debate tactic. It goes something like:

1. Only plans to kill my sister's addiction that account for my sister's feelings will work.

2. Only my sister can fully account for my sister's feelings.

3. Therefore, only my sister can invent successful plans to kill her addiction.

4. As a corollary, anyone else's plans to kill my sister's addiction will fail.

It is known that heroin addicts invent good-looking plans for killing their addictions, but do not invent good plans for killing their addictions. By this argument she can ensure that all plans to kill her addiction will always eventually fail.

The epistemically correct response, even if it's not necessarily persuasive in this form (for otherwise I would have persuaded her), is to say that I don't actually need to experience what she has to come up with good plans for killing addictions. "Not knowing what it's like to be an addict doesn't make me bad at making decisions about addictions," pattern-matches to, "I don't empathize with you," and, if she really wasn't listening, "I claim to know more about your own phenomenal experiences than you."

Sometimes how someone feels really doesn't matter, in really specific cases. That is, sometimes it's not necessary for an argument to follow. If you let people conflate this specific and useful objection with a more general sort of paternalism where you always ignore the relevance of everyone's feelings, then you might flinch from being right or doing right.