Ohh yes indeed. Which is why there are courts, and laws, and social mores. Sex isn't a perfect libertarian utopia of mutual contract negotiation and enforcement, even in the world we're imagining in this thread.

it can likewise not mean anything if certain other formal power differences exist.

Incidentally, as an aside into VERY dangerous waters, I think this is the actual important distinction. Enthusiastic consent is only possible between equals or near-equals. When any formal power differences exist, it muddies the waters too much for society to trust that consent was enthusiastically given.

Which, interestingly enough, makes pedophilia a problem of power-disparity interfering with the consent mechanism, instead of a mere social 'ugh field', which is deeply satisfying as a rationalist. (It also means that these problems can be corrected for, but discussing that might drift too far into taboo even for this site).