It's an interesting question. My thoughts follow a similar path and I like the way you described it here:

My hypothesis is that "blackmail" is what the suggestion of your mind to not cooperate feels like from the inside, the answer to a difficult problem computed by cognitive algorithms you don't understand, and not a simple property of the decision problem itself.

Taking a step back from the internal viewpoint we can also give a workable description in social terms. Which way people will tend to think of the decision offered is primarily determined by social dominance. Apart from the relative status of the actors themselves the status of the threatened negative action makes a difference on whether someone thinks 'extortion'. Given an approximately equivalent payoff matrix in terms of utility two different scenarios could be categorised differently with respect to extortion because the decision maker instinctively associates various different levels of 'legitimacy'.