if a person claims to have a "belief" about something unempirical, they're wrong.

Why si that relevant? The question is how to explain belief:-

"To me it seem the problem here is simply trying to treat natural language sentences as real things when they are only an approximate abstraction, that breaks down in these kinds of edge cases.

There are no discrete "belief's" with "justifications", there are only a probability distribution over the configuration space of all possible histories of sensory input. And that's just another layer of abstraction really, but it's enough for now."

Wrong beliefs exist

A theory that explains why people sometimes say "two plus two equals five" is a theory of psychology

exactly.

similarly, a theory that explains why people sometimes say things like "there is an invisible dragon in my garage" is a theory of psychology and not a theory of epistemology.

Epistemology can't ignore falsehood.