But he isn't an unusually good one at all. He actively dislikes actually reading peoples minds.

The latter statement isn't evidence for the former. Harry dislikes broomstick riding as an activity, but is still naturally gifted at it, and successful on the occasions when he needs to do it.

Here is our best example of Voldemort talking about his abilities:

Then in fury I threw aside masks and caution, I used my Legilimency, I dipped my fingers into the cesspit of his stupidity and tore out the truth from his mind. I did not understand and I wanted to understand. With my command of Legilimency I forced his tiny clerk-brain to live out alternatives, seeing what his clerk-brain would think of Lucius Malfoy, or Lord Voldemort, or Dumbledore standing in my place.

That sounds pretty advanced to me, and the way he speaks of it ("with my command of Legilimency") suggests pride in his abilities as well.

Here is Moody talking about Voldemort's abilities, with Dumbledore listening and not disagreeing:

“And I’ll warn you of this but once. Voldie isn’t like any other Legilimens in recorded history. He doesn’t need to look you in the eyes, and if your shields are that rusty he’d creep in so softly you’d never notice a thing.”

We are never told what the wizarding public at large thinks of Voldemort's Legilimency powers, so I don't know where you're getting that argument.

And for the record, this is Mr Bester's assessment of Harry:

You may indeed be able to learn Occlumency at eleven years of age. This astounds me. I had thought Mr. Dumbledore was pretending to be insane again. Your dissociative talent is so strong that I am surprised to find no other signs of childhood abuse, and you may become a perfect Occlumens in time.

It's also worth noting that while Moody doesn't seem especially impressed at the power of Harry's Occlumency barriers, his only comment is that they are rusty, not that they provide insufficient protection for practical purposes.