"Can you imagine having to program using 100Hz CPUs, no matter how many of them you had?"
No, it would be very difficult. But one thing I'm wondering is what's the instruction set of the neuron? I'm probably taking the analogy too far. Is it more advanced then add/sub/mult/div ?
So how much of the brains advanced nature comes from slower processors with better instruction sets and how much comes from network effects (both spatial and temporal)?
FWIW, As far as caching goes, I've noticed cache failures many, many times in my life. Mostly when I'm doing something that's 99% routine but for some reason I should be changing that last 1% and forget to. For example, if I'm supposed to run an errand on the way home, it's not uncommon for me to forget the errand. I leave work, think the goal is home and pull the set route from my brain. In fact driving home is so rote that I often don't remember all the details of the drive. It's not uncommon to hit home and then realize I need to go to the grocery store that was on the way. I often think of "brain farts" like that as either a problem of my really broad decision tree (I often maximize a choice for the local branch I'm in and maximize across the whole tree). Hmm I smell gas, Hmm I need light, I know I'll light a match! Or as collisions in my memory hash table, like the aforementioned picking my normal driving routine rather than deviating when I was supposed to.