Actually, your metaphor is more apt than you give it credit for. Native speakers can't be 'proven wrong' in their use of the language as 1) no language has a formal grammar and 2) to the extent that there are rules they are extracted from the way native speakers use the language. Something like morality, then.
Come to think of it, this can be used to construct a pretty intuitive response to those that claim that 'without god there is no objective morality and therefore society will collapse'. There is no formal grammar for English, and yet we're able to communicate pretty well.
[another stray thought] ...so then CEV would be like trying to extract a fully formal grammar for a given language, only harder.
Native speakers can't be 'proven wrong' in their use of the language
Right, but they can be proven wrong in the explanations they give about their use of language (except for rare pathological sentences, speakers of the same language agree which sentences "feel wrong"). Disproving an explanation about one's morality is much harder.
(I don't know if I'm disagreeing with you here)
If you've spent any time with foreigners learning your language, you may have been in conversations like this:
People can't automatically state the rules underlying language, even though they follow them perfectly in their daily speech. I've been made especially aware of this when teaching French to Chinese students, where I had to frequently revise my explanation, or just say "sorry, I don't know what the rule is for this case, you'll just have to memorize it". You learn separately how to speak the language and how to apply the rules.
Morality is similar: we feel what's wrong and what's right, but may not be able to formulate the underlying rules. And when we do, we're likely to get it wrong the first time. For example you might say:
But unlike grammar, people don't always agree on right and wrong : if Alfred unintentionally harms Barry, Barry is more likely to think that what Alfred did was morally wrong, even if both started off with similar moral intuitions. So if you come up with an explanation and insist it's the definition of morality, you can't be "proven wrong" nearly as easily as on grammar. You may even insist your explanation is true, and adjust your behavior accordingly, as some religious fanatics seem to do ("what is moral is what God said" being a quite common rule people come up with to explain morality).
So: beware of your own explanations. Morality is a complex topic, you're even more likely to shoot yourself in the foot than with grammar, and even less likely to realize that you're wrong.
(edit) Related posts by Eliezer: Fake Justification, Fake Selfishness, Fake Morality.