A preprint of the "The errors, insights and lessons of famous AI predictions – and what they mean for the future" is now available on the FHI's website.

Abstract:

Predicting the development of artificial intelligence (AI) is a difficult project – but a vital one, according to some analysts. AI predictions are already abound: but are they reliable? This paper starts by proposing a decomposition schema for classifying them. Then it constructs a variety of theoretical tools for analysing, judging and improving them. These tools are demonstrated by careful analysis of five famous AI predictions: the initial Dartmouth conference, Dreyfus's criticism of AI, Searle's Chinese room paper, Kurzweil's predictions in the Age of Spiritual Machines, and Omohundro's ‘AI drives’ paper. These case studies illustrate several important principles, such as the general overconfidence of experts, the superiority of models over expert judgement and the need for greater uncertainty in all types of predictions. The general reliability of expert judgement in AI timeline predictions is shown to be poor, a result that fits in with previous studies of expert competence.

The paper was written by me (Stuart Armstrong), Kaj Sotala and Seán S. Ó hÉigeartaigh, and is similar to the series of Less Wrong posts starting here and here.

This post goes along with this one, which was merely summarising the results of the volunteer assessment. Here we present the further details of the methodology and results.

Kurzweil's predictions were decomposed into 172 separate statements, taken from the book "The Age of Spiritual Machines" (published in 1999). Volunteers were requested on Less Wrong and on reddit.com/r/futurology. 18 people initially volunteered to do varying amounts of assessment of Kurzweil's predictions; 9 ultimately did so.

Each volunteer was given a separate randomised list of the numbers 1 to 172, with instructions to go through the statements in the order given by the list and give their assessment of the correctness of the prediction (the exact instructions are at the end of this post). They were to assess the predictions on the following five point scale:

• 1=True, 2=Weakly True, 3=Cannot decide, 4=Weakly False, 5=False

They assessed a varying amount of predictions, giving 531 assessments in total, for an average of 59 assessments per volunteer (the maximum attempted was all 172 predictions, the minimum was 10). They generally followed the randomised order correctly - there were three out of order assessments (assessing prediction 36 instead of 38, 162 instead of a 172, and missing out 75). Since the number of errors was very low, and seemed accidental, I decided that this would not affect the randomisation and kept those answers in.

The assessments (anonymised) can be found here.

Carbon nanotubes: The weird world of 'remote Joule heating'

Minimizing Joule heating remains an important goal in the design of electronic devices^1, 2. The prevailing model of Joule heating relies on a simple semiclassical picture in which electrons collide with the atoms of a conductor, generating heat locally and only in regions of non-zero current density, and this model has been supported by most experiments. Recently, however, it has been predicted that electric currents in graphene and carbon nanotubes can couple to the vibrational modes of a neighbouring material^3, 4, heating it remotely⁵. Here, we use in situ electron thermal microscopy to detect the remote Joule heating of a silicon nitride substrate by a single multiwalled carbon nanotube. At least 84%of the electrical power supplied to the nanotube is dissipated directly into the substrate, rather than in the nanotube itself. Although it has different physical origins, this phenomenon is reminiscent of induction heating or microwave dielectric heating. Such an ability to dissipate waste energy remotely could lead to improved thermal management in electronic devices⁶."

Carbon nanotubes in biology and medicine: In vitro and in vivo detection, imaging and drug delivery