More than six years late, but better late than never...

'Concepts are utilized in reasoning to reduce and structure search space' - anyone have any references or ideas for further developments of this line of thought? Seems very interesting and related to the philosophical idea of abduction or inference to the best explanation. (Perhaps the relation is one of justification.)

Also, since I find the OP compelling despite this point, I would be interested to see how far they can be reconciled.

My guess, loosely expressed, is that the stuff in Eliezer's OP above about the importance of good bookkeeping to prevent update messages bouncing back is sound, and should be implemented in designing intelligent systems, but some additional, more abductionesque process could be carefully laid on top. And when interpreting human reasoning, we should perhaps try to learn to distinguish whether, in a given case of a non-predictive empirical belief, the credence comes from bad bookkeeping, in which case it's illegitimate, or an abductive process which may be legitimate, and indeed may be legitimated along the lines of Vladimir's tantalizing hint in the parent comment.