which depends on the audience.

Yeah, probably that's it. While I'm positive that among musicians just having a decent sense of rhythm and melody and improvising on the E flat minor pentatonic scale (AKA “only playing the black keys”) is lower status than having spent hundreds of hours taking piano lessons and rehearsing, I'm not at all sure whether it'd also be lower status among other people, and indeed now that I think about it, my model of non-musicians says it wouldn't.

If I can take your answer literally,

I only normally use the word “improvising” about playing an instrument, or occasionally about vernacular dance (just discovered this term, BTW).