= 27d567bc2d48d9f40e9ccc20ea370b15)
AshwinV comments on Debunking Fallacies in the Theory of AI Motivation - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (343)
That seems to me like an argument from lack of imagination. The fact that reinforcement learning is the best among those you can easily imagine doesn't mean that it's the best overall.
If reinforcement learning would be the prime way we learn, understanding Anki cards before you memorize them shouldn't be as important as it is. Having a card fail after 5 repetitions because the initial understanding wasn't deep enough to build a foundation suggests that learning is about more than just reinforcing. Creating the initial strong understanding of a card doesn't feel to me like it's about reinforcement learning.
On a theoretical level reinforcement learning is basically behaviorism. It's not like behaviorism never works but modern cognitive behavior therapy moved beyond it. CBT does things that aren't well explainable with behaviorism.
You can get rid of a phobia via reinforcement learning but it takes a lot of time and gradual change. There are various published principles that are simply faster.
Pigeons manage to beat humans at a monty hall problem: http://www.livescience.com/6150-pigeons-beat-humans-solving-monty-hall-problem.html The pigeons engage the problem with reinforcement learning which is in this case a good strategy. Human on the other hand don't use that strategy and get different outcomes. To me that suggest a lot of high level human thought is not about reinforcement learning.
Given our bigger brains we should be able to beat the pigeons or at least be as good as them when we would use the same strategy.
Uhm. Is there any known experiment that has been tried which has failed with respect to RL?
In the sense, has there been an experiment where one says RL should predict X, but X did not happen. The lack of such a conclusive experiment would be somewhat evidence in favor of RL. Provided of course that the lack of such an experiment is not due to other reasons such as inability to design a proper test (indicating a lack of understanding of the properties of RL) or lack of the experiment happening to due to real world impracticalities (not enough attention having been cast on RL, not enough funding for a proper experiment to have been conducted etc.)
In general scientists do a lot of experiments where they make predictions about learning and those predictions turn out to be false. That goes for predictions based on RL as well as prediction based on other models.
Wikipedia describes RL as:
Given that's an area of machine learning you usually don't find psychologists talking about RL. They talk about behaviorism. There are tons of papers published on behaviorism and after a while the cognitive revolution came along and most psychologists moved beyond RL.
Not quite true, especially not if you count neuroscientists as psychologists. There have been quite a few papers by psychologists and neuroscientists talking about reinforcement learning in the last few years alone.
It appears to me that ChristianKI just listed four. Did you have something specific in mind?
Uhm, I kind of felt the pigeon experiment was a little misleading.
Yes, the pigeons did a great job of switching doors and learning through LR.
Human RL however (seems to me) takes place in a more subtle manner. While the pigeons seemed to focus on a more object level prouctivity, human RL would seem to take up a more complicated route.
But even that's kind of besides the point.
In the article that Kaj had posted above, with the Amy Sutherland trying the LRS on her husband, it was an interesting point to note that the RL was happening at a rather unconscious level. In the monty hall problem solving type of cognition, the brain is working at a much more conscious active level.
So it seems more than likely to me that while LR works in humans, it gets easily over-ridden if you will by conscious deliberate action.
One other point is also worth noting in my opinion.
Human brains come with a lot more baggage than pigeon brains. Therefore, it is more than likely than humans have learnt not to switch through years of re-enforced learning. It makes it much harder to unlearn the same thing in a smaller period of time. The pigeons having lesser cognitive load may have a lot less to unlearn and may have made it easier for them to learn the switching pattern.
Also, I just realised that I didn't quite answer your question. Sorry about that I got carried away in my argument.
But the answer is no, I don't have anything specific in mind. Also, I don't know enough about things like what effects RL will have on memory, preferences etc. But I kind of feel that I could design an experiment if I knew more about it.