...Neural nets are even better than that, because they enforce a mostly continous/differentiable energy landscape which helps inference/optimization.

I wonder whether this is a general property or is the success of continuous methods limited to problem with natural continuous models like vision.

Deep nets (of various forms) automatically share submodel components AND subcomputations/subexpressions amongst those submodels.

Yes, this is probably important.

First, due to nonlinear foveation your visual system can only read/parse a couple of words/symbols during each saccade - only those right in the narrow center of the visual cone, the fovea. So it takes a number of clock cycles or steps to scan the entire page, and your brain only has limited working memory to put stuff in.

Scanning the page is clearly not the bottleneck: I can read the page much faster than solve the exercises. "Limited working memory" sounds a claim that higher cognition has much less computing resources than low level tasks. Clearly visual processing requires much more "working memory" than solving a couple of dozens of exercises in arithmetic. But if we accept this constraint then does the brain still qualify for a ULM? It seems to me that if there is a deficiency of the brain's architecture that prevents higher cognition from enjoying the brain's full power, solving this deficiency definitely counts as an "architectural innovation".