Is the full paper on a repository? I'd like to cite it for an assignment and there's an explicit prohibition on using blog posts.
A second lesson could be about the importance of philosophy: it does seem that philosophers' meta-analytical skills can contribute useful ideas to AI - a fact that is certainly not self-evident.
You mean, pointing out the inductive/linear extrapolation/planning/confirmation fallacies? That's no longer in the realm of philosophy, but rather cognitive science.
I read one of his books a long time ago, and the fact that he was basically saying that playing chess was fundamentally too difficult for a computer went a long way in convincing me that we overestimate the magic that happens inside human brains.
He's overconfident, as are most experts - "computers will never" is a stupid prediction. But his reasons why the computer chess-playing machines of the 60s wouldn't work were correct. The problem in 1965 was that people were underestimating the difficulty of what went on in the human brain.
Dreyfus is a professional naysayer, so I am not sure what there is to learn from him if you want to predict the future well. His output is a constant function (or if you prefer he is the proverbial "stopped clock".) In fact, it is very difficult for a naysayer to make non-obvious testable predictions that aren't overtaken by progress. I doubt Dreyfus in the 60s thought we would have e.g. self-driving cars, or that such a thing was even possible.
When I read his paper, I was impressed. He may be a professional naysayer, but in 1965, he was the most correct predictor for AI around - and generally for the right reasons.
I once took a machine learning class, where one of the assignments was to construct a classifier that would assign people into one of two classes, A or B, based on some characteristics. The dataset we got had the property that class A occurred about 75% of the time. One of the points the instructor mentioned is that no matter how fancy a classification algorithm you use, it has to be able to beat the "baseline predictor" which just outputs A for everyone, and thus gets 75% accuracy via a constant time algorithm.
I certainly agree that Dreyfus was mostly right in 1965, but I thought the point of these case studies is learning lessons for predicting the future accurately ourselves. The point with Dreyfus is that he is a "baseline predictor." He always outputs A. There is nothing interesting to learn from him.
Again, the fact that he got the reasons right (the hype cycle, the fact human chess players were performing very differently from what the AI designers were doing, etc...) lifts him up a bit. I don't know what he's been up to since then, though.
Can we find a good baseline predictor today who's performing as well?
Can you present the arguments for this (at the level of Dreyfus in http://www.rand.org/content/dam/rand/pubs/papers/2006/P3244.pdf )?
Sorry, I may not be explaining myself very well. I agree that Dreyfus is quite smart, and writes well. I also agree that he may have had good arguments against AI progress in the 1960s. But I don't agree that this is how you should evaluate prophets. Prophets are doing prediction -- a standard statistical problem. The way to evaluate predictors is on datasets. A single prediction success, no matter how eloquent, is not really proof of the efficacy of the prophet.
If, on the other hand, a prophet consistently applies an algorithm and predicts correctly, well that becomes interesting and worthy of further study. The modern day prophet is Nate Silver, a humble statistician. He said recently that he is uncomfortable with his fame because all he is doing is simple statistical models + a little special sauce relevant to the domain. So my question to you is this: in what way can you improve your prediction algorithm by studying Dreyfus?
You ask him: "when will the singularity happen?" "When will a machine pass the Turing test?" "When will machines do this or that?" His answer for anything not trivially possible is "never." Naysaying is "boring," algorithmically.
I don't think Dreyfus is a generally good prophet. I think he made a great prediction in 1965, and that it would have been hard to see at the time that it was a good prediction. The lessons to draw, in my opinion, were "sometimes outsiders have very correct predictions", and "some of the features of Dreyfus's predictions (the specific examples, decomposition and understanding) are (weak) signs of good predictive ability".
Ignore Dreyfus himself for the moment. A paper was published that made correct predictions, and gave correct explanations for them, at a time when most experts in the field disagreed. The question is, was there a better cognitive strategy or a better prediction algorithm those experts could have followed, which would have allowed them to recognize the rightness of that paper?
"Never" is not a testable prediction. Break down predictions into finite-time-horizon groups and judge each against the baseline of "nothing happens in the next n years".
Much later edit: IlyaShpitser has correctly pointed out that my comment makes no sense.
Of course "never" is testable. The way to falsify is to exhibit a counterexample. "Human beings will never design a heavier than air flying machine" (Lord Kelvin, 1895), "a computer will never beat the human world champion in chess," etc. All falsified, therefore, all testable. If anything, an infinite horizon statement like "never" is more vulnerable to falsification, and therefore should get more "scientific respect."
It's only testable in one direction -- if you like, "never" is testable but "ever" isn't. I don't have a formal argument to hand, but it seems vaguely to me that a hypothesis preferably-ought to be falsifiable both ways.
Myself, Kaj Sotala and Seán ÓhÉigeartaigh recently submitted a paper entitled "The errors, insights and lessons of famous AI predictions and what they mean for the future" to the conference proceedings of the AGI12/AGI Impacts Winter Intelligenceconference. Sharp deadlines prevented us from following the ideal procedure of first presenting it here and getting feedback; instead, we'll present it here after the fact.
The prediction classification shemas can be found in the first case study.
Dreyfus's Artificial Alchemy
Hubert Dreyfus was a prominent early critic of Artificial Intelligence. He published a series of papers and books attacking the claims and assumptions of the AI field, starting in 1965 with a paper for the Rand corporation entitled 'Alchemy and AI' (Dre65). The paper was famously combative, analogising AI research to alchemy and ridiculing AI claims. Later, D. Crevier would claim ''time has proven the accuracy and perceptiveness of some of Dreyfus's comments. Had he formulated them less aggressively, constructive actions they suggested might have been taken much earlier'' (Cre93). Ignoring the formulation issues, were Dreyfus's criticisms actually correct, and what can be learned from them?
Was Dreyfus an expert? Though a reasonably prominent philosopher, there is nothing in his background to suggest specific expertise with theories of minds and consciousness, and absolutely nothing to suggest familiarity with artificial intelligence and the problems of the field. Thus Dreyfus cannot be considered anything more that an intelligent outsider.
This makes the pertinence and accuracy of his criticisms that much more impressive. Dreyfus highlighted several over-optimistic claims for the power of AI, predicting - correctly - that the 1965 optimism would also fade (with, for instance, decent chess computers still a long way off). He used the outside view to claim this as a near universal pattern in AI: initial successes, followed by lofty claims, followed by unexpected difficulties and subsequent disappointment. He highlighted the inherent ambiguity in human language and syntax, and claimed that computers could not deal with these. He noted the importance of unconscious processes in recognising objects, the importance of context and the fact that humans and computers operated in very different ways. He also criticised the use of computational paradigms for analysing human behaviour, and claimed that philosophical ideas in linguistics and classification were relevant to AI research. In all, his paper is full of interesting ideas and intelligent deconstructions of how humans and machines operate.
All these are astoundingly prescient predictions for 1965, when computers were in their infancy and their limitations were only beginning to be understood. Moreover he was not only often right, but right for the right reasons (see for instance his understanding of the difficulties computer would have in dealing with ambiguity). Not everything Dreyfus wrote was correct, however; apart from minor specific points (such as his distrust of heuristics), he erred most mostly by pushing his predictions to extremes. He claimed that 'the boundary may be near' in computer abilities, and concluded with:
Currently, however, there exists 'digital automata' that can beat all humans at chess, translate most passages to at least an understandable level, and beat humans at 'Jeopardy', a linguistically ambiguous arena (Gui11). He also failed to foresee that workers in AI would eventually develop new methods to overcome the problems he had outlined. Though Dreyfus would later state that he never claimed AI achievements were impossible (McC04), there is no reason to pay attention to later re-interpretations: Dreyfus's 1965 article strongly suggests that AI progress was bounded. These failures are an illustration of the principle that even the best of predictors is vulnerable to overconfidence.
In 1965, people would have been justified to find Dreyfus's analysis somewhat implausible. It was the work of an outsider with no specific relevant expertise, and dogmatically contradicted the opinion of genuine experts inside the AI field. Though the claims it made about human and machine cognition seemed plausible, there is a great difference between seeming plausible and actually being correct, and his own non-expert judgement was the main backing for the claims. Outside of logic, philosophy had yet to contribute much to the field of AI, so no intrinsic reason to listen to a philosopher. There were, however, a few signs that the paper was of high quality: Dreyfus seemed to be very knowledgeable about progress and work in AI, and most of his analyses on human cognition were falsifiable, at least to some extent. These were still not strong arguments to heed the skeptical opinions of an outsider.
The subsequent partial vindication of the paper is therefore a stark warning: it is very difficult to estimate the accuracy of outsider predictions. There were many reasons to reject Dreyfus's predictions in 1965, and yet that would have been the wrong thing to do. Blindly accepting non-expert outsider predictions would have also been a mistake, however: these are most often in error. One general lesson concerns the need to decrease certainty: the computer scientists of 1965 should at least have accepted the possibility (if not the plausibility) that some of Dreyfus's analysis was correct, and they should have started paying more attention to the 'success-excitement-difficulties-stalling' cycles in their field to see if the pattern continued. A second lesson could be about the importance of philosophy: it does seem that philosophers' meta-analytical skills can contribute useful ideas to AI - a fact that is certainly not self-evident.
References: