I think a question such as "Do you actually like this music?" carries a presumed shared understanding that it's quite possible and reasonable that the interlocutor doesn't like the music, but came to the concert for other reasons, and the speaker inferred this from something about him/her.

E.g. someone is expected to be at the concert anyway (because they're a friend of a band member, say), and they know you know it, so you ask them "do you actually like the music?". Or, you see that someone came as part of a large group but isn't wild with excitement like the rest of their group, so you suspect they went out of a social obligation, and they understand that people might suspect that, looking at them.

In the LW meetup scenario, the question with 'actually' might be unraveled as "We both know that this is a meetup for people who read LW, and we also both understand that, based on your gender and appearance, which is all I know about you, you're not a likely demographic; so is it out of social obligation that you came here?" Whether the recipient will be offended depends a lot on her character, mood, whether she will want to make an allowance for poor social skills of her interlocutor, etc. I think it's both not maliciously meant and a rather rude thing to say.

Here's another analogy: imagine that it's a chess club meeting and the question is "Do you actually play chess, or did someone drag you here?"