I read that. I agree with the argument. But it doesn't really address my intuition behind my argument.
The idea is that you have concurrent processes creating partial models of partial but overlapping aspects of reality. These models a) help making predictions for each aspect (descriptively), b) may help acting in the context of the aspect (operational/prescriptively) and c) may be on the symbolic layer inconsistent.
Do you want to kick out all the benefits to gain consistency? It could be that you can't achieve consistency of overlapping models at all without some super all encompassing model. Or it could be that such a super-model is horribly big and slow.
If we're going to be building a Seed AI, I really don't think a good design would involve the AI reasoning using multiple, partially overlapping, possibly inconsistent models, especially since I'm not sure how the AI would go about updating those models if it made contradictory observations. For example, upon receiving contradictory evidence, which of its models would it update? One? Two? All of them? If you decide to work with ad hoc hypotheses that contradict not only reality, but each other, just because it's useful to do so, the price you pay is throwi...
Another month, another rationality quotes thread. The rules are: