Not sure the status implication is accurate. When I vote down someone high-status, I don't feel any particular compulsion to explain myself. If anything, it makes me anticipate that I'm unlikely to change anyone's mind.

I think a much closer analogy than saying "you fail" is frowning.

Would you prefer that I posted a lot of comments starting with "I voted this down because", or that I didn't vote on comments I think detract from the site?