Does that make a difference to your point?

Yes. The difference in perspective probably explains why Eliezer thought Less Wrong was a good name, whereas you do not. Do not compare yourself to others; "The best physicist in ancient Greece could not calculate the path of a falling apple."

Indeed, a valuable point. So what's up with the score keeping system of LW then. It encourages thinking in terms of sides and competition. -1, not my side, +1 my side. -1 lost, +1 won.

It's a hurdle to get past thinking of it in that way for some people, to be sure. It seems a worthwhile cost though, for an easy way to efficiently express approval/disapproval of a comment, combined with automatic hiding of really bad comments from casual readers.

While some people use them that way, voting should not generally be used to mean "I agree" or "I disagree". The preferred interpretation is "I would like to see [more/fewer] comments like this one" (which may yet include agreement/disagreement, but they should be minor factors as compared to quality).