The new paper by Stuart Armstrong (FHI) and Kaj Sotala (SI) has now been published (PDF) as part of the Beyond AI conference proceedings. Some of these results were previously discussed here. The original predictions data are available here.
Abstract:
This paper will look at the various predictions that have been made about AI and propose decomposition schemas for analysing them. It will propose a variety of theoretical tools for analysing, judging and improving these predictions. Focusing specifically on timeline predictions (dates given by which we should expect the creation of AI), it will show that there are strong theoretical grounds to expect predictions to be quite poor in this area. Using a database of 95 AI timeline predictions, it will show that these expectations are born out in practice: expert predictions contradict each other considerably, and are indistinguishable from non-expert predictions and past failed predictions. Predictions that AI lie 15 to 25 years in the future are the most common, from experts and non-experts alike.
The original data can be found via: http://lesswrong.com/lw/e79/ai_timeline_prediction_data/
(much better to use that than to squint at the pictures!)
My subjective impressions: predictors very rarely quote or reference each other when making predictions. Many predictions seem purely an individual guess. I've seen no sign of an expert consensus, or of much experts critiquing or commending each other's work. I really feel that predicting AI has not been seen as something where anyone should listen to other people's opinions. There are some exceptions - Kurzweil, for instance, seems famous enough that people are willing to quote his estimates, usually to claim he got it wrong - but too few.
I'm actually thinking that the "non-experts were no better than experts" bit is maybe a little misleading, as I remember seeing a lot of the non-experts base their predictions on what experts had been saying.