[I'd put this in an open thread, but those don’t seem to happen these days, and while this is a quote it isn't a Rationality Quote.]
You know, one of the really weird things about us human beings […] is that we have somehow created for ourselves languages that are just a bit too flexible and expressive for our brains to handle. We have managed to build languages in which arbitrarily deep nesting of negation and quantification is possible, when we ourselves have major difficulties handling the semantics of anything beyond about depth 1 or 2. That is so weird. But that's how we are: semantic over-achievers, trying to use languages that are quite a bit beyond our intellectual powers.
— Geoffrey K. Pullum, Language Log, “Never fails: semantic over-achievers”, December 1, 2011
This seems like it might lead to something interesting to say about the design of minds and the usefulness of generalization/abstraction, or perhaps just a good sound bite.
Your brain stores memories of input and also of previous thoughts you had and the experience of taking actions. Within the “replaced with a new version” view of the time evolution of your brain (which is also the pure-functional-programming view of a process communicating with the outside world), we can say that the input it receives next iteration contains lots of information from outputs it made in the preceding iteration.
But with the reinforcement learning algorithm, the previous outputs are not given as input. Rather, the previous outputs are fed to the reward function, and the reward function's output is fed to the gradient descent process, and... (read more)