The knowledge or belief in moral realism is acquired. People may be born with moral realist intuitions, but they are not born with coherent arguments in favor of moral realism. And no-one has the right to just believe something without proof.
You can replace the phrase "moral realist" with "physical realist" in the above statement and your subsequent argument and it remains equally valid.
What exactly do you mean by 'physical realism'? At first I thought it was something like the simple claim that "the physical world objectively exists independently of us", or maybe like positivism. But googling 'physical realism' brings up mostly pseudoscientific nonsense, so it may not be a commonly used term, and there are no wikipedia/Stanford/etc. entries. So I wanted to make sure what you meant by it.
Here's the new thread for posting quotes, with the usual rules: