Let's say you want to seed an AGI with knowledge.
Some humans get some gene therapy that turns their blood blue. Then the AGI reasons that those entities aren't humans anymore and therefore don't need to be treated according to the protocol for treating humans. Or it get's confused with other issues about 'red'.
The only way around this is to specifically tell the FAI the context in which the statement is true. If you look at Eliezers post about truth you won't find a mention that context is important and needs to be passed along.
It's just not there. It's not a central part of his idea of truth. Instead he talks about probabilities of things being true.
None of my Anki cards has "this is true with 0.994%" probability. Instead it has a reference to context. That's because context is more important than probability for most knowledge. In the framework that Eliezer propagates probability is of core importance.
That isn't an example where a truth value depends on context, it's an example where making the correct deductions depends on the correct theoretical background.
However I agree that quantifying what you haven't first understood is pointless.
As per a recent comment this thread is meant to voice contrarian opinions, that is anything this community tends not to agree with. Thus I ask you to post your contrarian views and upvote anything you do not agree with based on personal beliefs. Spam and trolling still needs to be downvoted.