Which brings up the question: suppose that your values are defined in terms of an ontology which is not merely false but actually logically inconsistent, though in a way that is too subtle for you to currently grasp. Is it rational to try to learn the logical truth, and thereby lose most or all of what you value? Should we try to hedge against such a possibility when designing a friendly AI? If so, how?

You do not lose any options by gaining more knowledge. If the optimal response to have when your values are defined in terms of an inconsistent ontology is to go ahead and act as if the ontology is consistent then you can still choose to do so even once you find out the dark secret. You can only gain from knowing more.

If your values are such that they do not even allow a mechanism for creating an best effort approximation of values in the case of ontological enlightenment then you are out of luck no matter what you do. Even if you explicitly value ignorance of the fact that nothing you value can have coherent value, the incoherency of your value system makes the ignorance value meaningless too.

Should we try to hedge against such a possibility when designing a friendly AI? If so, how?

Make the most basic parts of the value system in an ontology that has as little chance as possible of being inconsistent. Reference to actual humans can ensure that a superintelligent FAI's value system will be logically consistent if it is in fact possible for a human to have a value system defined in a consistent ontology. If that is not possible then humans are in a hopeless position. But at least I (by definition) wouldn't care.