Hostile magic is obviously not the same as Legilmancy

It's also not the same as the sort of LifeAlert wards Quirrell (claims that he) put on Draco, which we know (that Quirrell claimed that) Heh specifically forbade.

Why would Dumbledore care and what would Dumbledore do differently if Dumbledore found out that Quirrell had looked into Hermione's mind?

Well, for one thing, he might look at the Map again. (He suspects Voldemort is trying to influence Hermione's mind; the Defence Prof is caught trying to read Hermione's mind; it doesn't take a genius.)

The fact that the eye contact was specifically mentioned seems important

But it wasn't, though. "The pale blue eyes watched her intently" is not an unambiguous indication of eye contact.

in my mind downvoting should be used for punishing (or expressing disapproval of) immoral things rather than of mistakes

How astonishingly strange. (I don't agree.)

People should probably be more laid back about downvoting.

Yes. People should. (Hint hint.)