Predictions of the future rely, to a much greater extent than in most fields, on the personal judgement of the expert making them. Just one problem - personal expert judgement generally sucks, especially when the experts don't receive immediate feedback on their hits and misses. Formal models perform better than experts, but when talking about unprecedented future events such as nanotechnology or AI, the choice of the model is also dependent on expert judgement.

Ray Kurzweil has a model of technological intelligence development where, broadly speaking, evolution, pre-computer technological development, post-computer technological development and future AIs all fit into the same exponential increase. When assessing the validity of that model, we could look at Kurzweil's credentials, and maybe compare them with those of his critics - but Kurzweil has given us something even better than credentials, and that's a track record. In various books, he's made predictions about what would happen in 2009, and we're now in a position to judge their accuracy. I haven't been satisfied by the various accuracy ratings I've found online, so I decided to do my own.

Some have argued that we should penalise predictions that "lack originality" or were "anticipated by many sources". But hindsight bias means that we certainly judge many profoundly revolutionary past ideas as "unoriginal", simply because they are obvious today. And saying that other sources anticipated the ideas is worthless unless we can quantify how mainstream and believable those sources were. For these reasons, I'll focus only on the accuracy of the predictions, and make no judgement as to their ease or difficulty (unless they say things that were already true when the prediction was made).

Conversely, I won't be giving any credit for "near misses": this has the hindsight problem in the other direction, where we fit potentially ambiguous predictions to what we know happened. I'll be strict about the meaning of the prediction, as written. A prediction in a published book is a form of communication, so if Kurzweil actually meant something different to what was written, then the fault is entirely his for not spelling it out unambiguously.

One exception to that strictness: I'll be tolerant on the timeline, as I feel that a lot of the predictions were forced into a "ten years from 1999" format. So I'll estimate the prediction accurate if it happened at any point up to the end of 2011, if data is available. 

The number of predictions actually made seem to vary from source to source; I used my copy of "The Age of Spiritual Machines", which seems to be the original 1999 edition. In the chapter "2009", I counted 63 prediction paragraphs. I then chose ten numbers at random between 1 and 63, and analysed those ten predictions for correctness (those wanting to skip directly to the final score can scroll down). Seeing Kurzweil's nationality and location, I will assume all prediction refer only to technologically advanced nations, and specifically to the United States if there is any doubt. Please feel free to comment on my judgements below; we may be able to build a Less Wrong consensus verdict. It would be best if you tried to reach your own conclusions before reading my verdict or anyone else's. Hence I present the ten predictions, initially without commentary:

  • Prediction 5: Cables are disappearing. Communication between components, such as pointing devices, microphones, displays, printers and the occasional keyboard, uses short-distance wireless technology.
  • Prediction 7: The majority of text is created using continuous speech recognition (CSR) dictation software, but keyboards are still used. CSR is very accurate, far more so than the human transcriptionists who were used up until a few years ago.
  • Prediction 8: Also ubiquitous are language user interfaces (LUIs) which combine CSR and natural language recognition. For routine matters, such as simple business transactions and information inquiries, LUIs are quite responsive and precise. They tend to be narrowly focused, however, on specific types of tasks. LUIs are frequently combined with animated personalities. Interacting with an animated personality to conduct a purchase or make a reservation is like talking to a person using video conferencing, except the person is simulated.
  • Prediction 18: In the twentieth century, computers in schools were mostly on the trailing edge, with most effective learning from computers taking place in the home. Now in 2009, while schools are still not on the cutting edge, the profound importance of the computer as a knowledge tool is widely recognised. Computers play a central role in all facets of education, as they do in other spheres of life.
  • Prediction 20: Students of all ages typically have a computer of their own, which is a thin tabletlike device weighing under a pound with a very high resolution display suitable for reading. Students interact with their computers primarily by voice and by pointing with a device that looks like a pencil. Keyboards still exist, but most textual language is created by speaking. Learning materials are accessed through wireless communication.
  • Prediction 26: Print-to-speech reading devices for the blind are now very small, inexpensive, palm-sized devices that can read books (those that still exist in paper form) and other printed documents, and other real-world text such as signs and displays. These reading systems are equally adept at reading the trillions of electronic documents that are instantly available from the ubiquitous wireless worldwide network.
  • Prediction 29: Computer-controlled orthotic devices have been introduced. These "walking machines" enable paraplegics to walk and climb stairs. The prosthetic devices are not yet usable by all paraplegic persons, as many physically disabled persons have dysfunctional joints from years of disuse. However, the advent of orthotic walking systems is providing more motivations to have these joints replaced.
  • Prediction 44: Intelligent roads are in use, mainly for long-distance travel. Once your car's guidance system locks into the control sensors on one these highways, you can sit back and relax. Local roads, though, are still predominantly conventional.
  • Prediction 48: There is continuing concern with an underclass that the skill ladder has left far behind. The size of the underclass appears to be stable, however. Although not politically popular, the underclass is politically neutralised through public assistance and the generally high level of affluence.
  • Prediction 53: Beyond musical recordings, images, and movie videos, the most popular type of digital entertainment object is virtual experience software. These interactive virtual environments allow you to go whitewater rafting on virtual rivers, to hang-glide in a virtual Grand Canyon, or to engage in intimate encounters with your favourite movie star. Users also experience fantasy environments with no counterpart in the physical world. The visual and auditory experience of virtual reality is compelling, but tactile interaction is still limited.

 

Verdict

My scale for judging the predictions is: true, weakly true, weakly false, false.

Prediction 5: My office and the computer I'm typing on seem pretty full of cables. Nevertheless, it is true there has been a rise in wireless technology, and wireless computer components, even if they're not ubiquitous. I'll grade this as a weakly true.

Prediction 7: I have failed to find proper data for the first prediction. Anecdotally, it certainly seems false - keyboards are still in ubiquitous use, and I've never personally seen anyone use voice recognition to write documents of any length or even to send texts (a few personal experiments with Siri notwithstanding). The second claim in false: according to an assessment by the National Institute of Standards and Technology, the accuracy of CSR is still nowhere near surpassing human transcription. This leads extra credence to the first claim being false as well: without the diminished error rate, it's very hard to see CSR being used for the majority of text creation. False.

Prediction 8: Apart from the belief that the animated personality would be visual, this is a near-perfect description of Siri and similar assistants. The term "ubiquitous" is tricky, but if we interpret it to mean "to be found everywhere" (rather than "everyone has one"), then the prediction is weakly true (knocked down from true because of the uncertainty about ubiquity).

Prediction 18: Without needing to do the research, I think we can take this claim as evidently true.

Prediction 20: All the stuff about voice recognition is false. The only device that fits that description today is the smartphone, which has not achieved penetration of more than 50% among teenagers in 2011 (teenagers are the median "students of all ages"; adding in university students as well as pre-teens should lower the proportion, not raise it). "Learning materials are accessed through wireless communication" is hard to interpret, as it doesn't give any estimate to what proportion of learning material we are talking about. So though we can give Kurzweil kudos for imagining something like the smartphone, the prediction is weakly false.

Prediction 26: One can quibble about inexpensive, as the products seem to be in the $600 range, but those products certainly exist for book and magazine reading (though not for most signs and displays, as far as I can tell - certainly not in a form the blind can use). The second sentence is true for some screen readers, making the prediction essentially true.

Prediction 29: 2009 timeline wrong, but true in later years.

Prediction 44: The relative quantifier in the last sentence ("though, are still predominantly conventional") makes it clear that we should expect intelligent highways to be common among long-distance highways - this isn't a few experimental roads we're talking about. Though we have a few self-driving cars, we have nothing like the intelligent roads implied in this prediction, which specifically implies that most cars on those roads will be self-driven. False.

Prediction 48: The first part of the prediction is true. The second sentence seems false, whether one measures the underclass through relative income (where inequality has been increasing) or through an absolute standard of educational attainment (where the various graduating rates have gone up, implying the underclass is decreasing). There are other ways one could measure the underclass, giving different results. Since one could read the underclass as increasing or decreasing, should we take Kurzweil's claim that it is stable as the correct mean? No. All that means is that had he spelt out his claim in more detail at the time, it would likely have ended up false. Ambiguity does not make a false statement true. The last sentence is virtually impossible to confirm or infirm, so the whole prediction is weakly true and weakly false.

Prediction 53: This is a tricky one. The Wii and similar game consoles seem to fit the bill to some extent. However the tone suggests he is talking about a virtual reality experience, which is not what we currently have. So, does he mean virtual reality, or does he mean "games like what they had in 1999, except with much better graphics and features"? How would someone at the time have read the prediction? Again, ambiguity cannot be used to make a false statement true. I'm going to work on the assumption that had he merely meant "graphics and features of video games will improve a lot", he would have said so (certainly his prediction seems to promise much more than that). So the prediction is false.

But what if he was talking about modern games? For a start, his initial sentence gets the relative size of the industries wrong (though that can be read as a throw-away statement rather than a prediction). He also doesn't consider things like Facebook games, which make up a large part of the games industry, and are certainly not interactive virtual environments. What about "these virtual environments allow..."? Well, the statement is possibly an utter triviality, claiming that games exist which feature rafting, hang-gliding or erotic situations (that was already true in 1999). Or it claims that features like these are a major component of the most most popular games today, which is false (now, if he'd said "blowing things up with a marvellous amount of weapons..."). Fantasy environment is a much more common feature, so, I'm taking that as correct. Under this interpretation, the prediction is weakly true and weakly false for games. In total, reading the statement either way, I'll classify it as (contentiously) weakly false.

Note: I did read Kurzweil's assessment of his own predictions, after I had conducted my own analysis. In that assessment, nearly every ambiguous clause is interpreted in Kurzweil's favour. This could be Kurzweil twisting the predictions in his direction; it could be a blatant example of hindsight bias; or it could be that what Kurzweil meant to say was different from what he wrote. Unfortunately, there is no way for us to tell, so we must make do with what was written and interpret it as best we can.

 

Analysis

So, out of the ten predictions, five are to some extent true, four are to some extent false, and one is unclassifiable (reading through the rest of the predictions, completely informally, these proportions seem roughly correct).

Now imagine Kurzweil as a predictor who gives predictions, each with independent probability p of bring true (alternately, assume that a fixed proportion p of the 63 predictions are true, and pretend 63 is high enough that we can treat p as continuous without much loss). If we start with a uniform prior on p between 0 and 1, then we can update given this data. Model prediction 48 as true or false with equal probability. Then the posterior must be proportional to (1-p)5p5 + (1-p)4p6:

This has a mean above 54%, which I'd say is excellent. A prediction record over 50% for a decade that included huge increases in computer power, September 11th and the great recession is intuitively a very good one. Alas there is no central repository of prediction records from various futurists, but in the absence of that, his track record certainly feels impressive. Don't let the hindsight bias blind you to how hard this was, and don't simply think of every prediction as binary: generally, there are far more ways for a prediction to be false than there are for them to be true.

On the other hand, if we look at Kurzweil's own ranking of the predictions he gave in the "Age of Spiritual Machines", he grades himself as having either 102 out of 108 or 127 out of 147 correct (with caveats that "even the predictions that were considered 'wrong' in this report were not all wrong"). I've plotted the lower 127/147≈0.86 accuracy on the above graph; that is very far from being a mean estimate (it's in the 99th percentile of the probability distribution). But let's give Kurzweil all we can: we'll reclassify the arguable prediction 53 as being true (posterior proportional to (1-p)4p6 + (1-p)3p7):

That is still not enough to make his accuracy estimate reasonable: his estimate is in the 96th percentile of the probability distribution. Let's be even more generous: let's reclassify the intermediate prediction 48 as also being true (posterior proportional to (1-p)3p7):

Those were very generous adjustments; changing two results is a lot from a sample of ten. But even with the most generous adjustments and taking Kurzweil's lowest estimate of his own accuracy, he is still extraordinarily overconfident: his estimate is in the 94th percentile of the probability distribution. For fun, I flipped another prediction from false to true: even then, his estimate is in the 81th percentile of the probability distribution (and recall that if we were rigorous about the timeline that Kurzweil claimed, at least one of the true prediction would be false).

So what can this tell us about Kurzweil as a futurist, and about the predictions he makes? Essentially two points stand out:

  1. He's most likely good at predicting.
  2. He's most likely overconfident, reluctant to admit his misses, and hence unlikely to update on his failures.

So I feel we should take Kurzweil's predictions as a good baseline, with much wider error bars and caveats, paying relatively less attention to those areas where we feel that being a good Bayesian updater becomes important. We should thus probably pay more attention to his models than to his interpretation of his models.

New to LessWrong?

New Comment
39 comments, sorted by Click to highlight new comments since: Today at 2:48 AM

You should have the predictions coded by blinded independent raters. Undergrads aren't too expensive, and you can default to Amazon's Mechanical Turk (although they may google the text) for even cheaper and faster subjects.

Also, a helpful control would be to give the independent raters predictions that resemble Kurzweil's but with the signs flipped. E.g. instead of saying "Technology X will be ubiquitous" one would say "Technology X will not be ubiquitous." When you add up the percentages who agree with each statement (with only one presented to each rater) I suspect you will get a total over 100%. Experiments with horoscopes, "psychic readers" and similar have shown like effects.

I don't think Prediction 18 is true for primary and secondary school.

The Prediction 8 "animated personality" bit is clearly wrong, and I don't think it should be cut off as 'less important' or 'trivial' if you weren't doing this elsewhere. If you're going to weight by importance then that should be done with blind rating too.

Regarding Prediction 29, scoring as "true" something developed soon after the specified date [EDIT: seems fishy to me]. When predictions are only 10 years out for something widely agreed to be inevitable, good timing is the only impressive part of a prediction.

That's a good idea - will do that when I get back.

Regarding Prediction 29, scoring as "true" something developed soon after the specified date contradicts your prior plan to only count successes BEFORE the predicted date. When predictions are only 10 years out for something widely agreed to be inevitable, good timing is the only impressive part of a prediction. If that is allowed to slip it undercuts the whole exercise.

I stated that I decided to be somewhat flexible on the timeline, specifically because it felt like a lot was being shoehorned into the "ten years from 1999" format.

Edited my comment to reflect this.

Prediction 8: Apart from the belief that the animated personality would be visual, this is a near-perfect description of Siri and similar assistants. The term "ubiquitous" is tricky, but if we interpret it to mean "to be found everywhere" (rather than "everyone has one"), then the prediction is somewhat true (knocked down from true because of the uncertainty about ubiquity).

I suggest that you are underestimating their ubiquity by focusing on Siri. Almost every large corporation has a phone tree where at least the initial stages are served by an AI using voice recognition, with "enter that umber on the keypad" as backup. Notice the reference to "routine business transactions".

These don't use any form of natural language recognition - they work by having very rigidly defined responses that they can interpret (ie "say 'one' for hard to recognize or easily obfuscated department").

If Kurzweil's self-calibration is too optimistic, does this imply that his newer predictions will tend to be less accurate?

A general point: given the time you spent on this, you could and should have graded more than 10 predictions.

Note: I did read Kurzweil's assessment of his own predictions, after I had conducted my own analysis. In that assessment, nearly every ambiguous clause is interpreted in Kurzweil's favour. This could be Kurzweil twisting the predictions in his direction; it could be a blatant example of hindsight bias; or it could be that what Kurzweil meant to say was different from what he wrote. Unfortunately, there is no way for us to tell, so we must make do with what was written and interpret it as best we can.

If it makes you feel better, back when he assessed his predictions, I downloaded the PDF and remarked that he was clearly favoring himself and was often quietly redefining anything which existed in the lab or as doomed commercial products as widespread successes per his overall mood (and occasionally specific wording, as in the howler of voice transcription dominating typing).

It seems to me you're grading prediction 5 quite generously. Wireless technology is common, but not ubiquitous. Wireless keyboards and printers exist, but are far less common than wired. Pointing devices are (in my anecdotal experience) higher, but not by a lot; microphones are probably more commonly wireless than not. Displays, however, are all wired with only some exotic tech demos as wireless, as far as I can tell. I'm not sure how to interpret "disappearing", so I'll go with the less-ambiguous second sentence: "Communication between components ... uses short-distance wireless technology." Of the five technologies listed, it seems to me that he's unambiguously right about one (microphones), wrong about one (displays), and mostly wrong about three. Taking this track record and calling the overall prediction "mostly right", which is how I interpret your "partially true" seems like running straight into the conjunction fallacy. Each added detail is a further burden of proof, and this time Kurzweil gave himself four burdens too many.

From what I've seen, desktop microphones are still largely wired headsets, for best performance. Wireless microphones seem to be primarily bluetooth headsets focused on phones. Even then, I'd say blue tooth is less than 50% of the cell phone usage I observe on a daily basis, but that's anecdotal.

I'd say that the prediction obviously doesn't include integrated anything. Therefore I'd count desktop microphones, but not ones integrated into a cell phone or laptop. I'd count bluetooth cell phone mics, wired mics attached to webcams, but not laptop-integrated mics. I suspect more people have bluetooth headsets than wired desktop mics, but I don't have data to back that up.

Every MP3 player I've ever bought came with a WIRED pair of external earbuds, so I can be pretty confident that there are more wired than wireless earbuds out there.

My experience shopping for a headset for my desktop suggests that wired headsets are still vastly more common than wireless ones.

It seems cellphones are the only exception, and only because of the exclusion of integrated microphones (I've never seen a cellphone with a wired headset). I can see excluding integrated microphones that aren't used, but I would say that at least 50% of cell phone usage I see is based on an integrated (and therefor NOT wireless) microphone.

In short, while the world is clearly moving towards wireless, the issue of power / battery life still seems to be a rather solid obstacle. I vastly prefer a wired keyboard and mouse, because unlike my roommate, I've never died in a video game when my mouse's battery suddenly died :)

Earbuds weren't on the list, so I was ignoring them. It seems unfair and highly open to bias to add devices that weren't on the list when evaluating the prediction. The prediction is vague enough as it is, without going that route. When considering integrated microphones, I would say that they are part of one thing, not connected components of a system. I'm aware that definition is non-obvious; however, I believe it is justified by context. The prediction is about "cables are disappearing"; an integrated microphone (or cell phone display, or cell phone keyboard) has no part that an end-user would identify as a cable. I can't believe that Kurzweil thought wireless would become so ubiquitous that it would be used within a cell phone, and I don't see anything in the prediction that I can read to mean that.

That said, I may be vastly underestimating the number of headsets with microphones sold for gaming. I don't do gaming with them. Availability heuristic and all that; it wouldn't surprise me much if they outnumbered bluetooth headsets.

And to add another example, my previous cell phone came with a wired lapel mic. I used it on only rare occasions.

My scale for judging the predictions is: true, partially true, partially false, false.

I dislike this scale. One could argue that "partially false" is better than "partially true".

replaced "partially" with "somewhat".

try "roughly" maybe?

Going for "weakly true" and "weakly false".

That doesn't really solve the problem. For clarity's sake, you probably ought use "mostly true" and "mostly false".

"Mostly" is too strong a qualifier.

If you're looking for something that means "more than half", then "mostly" is a good choice. If you're looking for something that means "less than half", then you have the problem that " true" means less true than " false".

I scored Kurzweil 4/10, before reading Stuart's analysis, and thought I was being generous. 7, 8, 20, 44, 48, and 53 are all false, and the others are questionable. Satt's analysis is right on. Every paragraph paints a picture of the future that turned out to be overly optimistic.

When Kurzweil is right, he is barely right. Schools pay lip-service to the importance of computers now, but computer education is still lacking. Wireless devices are more common, but my desk is still a tangled mess.

Also notice how he gets the little details wrong. Now that we have touchscreen computers, we interface using our fingers, not styluses. Once cars finally drive themselves, it will be due to smart cars, not smart roads. Why would he even include these details?

Great post. Arnold Kling has a good discussion of Kurzweil's predictions somewhere, but I haven't been able to find it by Googling.

I agree that Kurzweil did well, making a significant number of specific, non-obvious correct predictions. But how well does Kurzweil's ability here generalize to other predictions? Kurzweil was predicting developments in his own field 10 years into the future. He has an advantage that products often take >4 years to develop, and he has insider knowledge of what kind of products the big tech companies are talking about in-house. (So we could compare him to internal discussions of possible products at Microsoft or Apple, etc.).

Arnold Kling has a good discussion of Kurzweil's predictions somewhere, but I haven't been able to find it by Googling.

This post has a link to the article I assume you're referring to, but the article isn't there - the Wayback Machine does have a copy of it though (no idea if this link which is supposed to point directly to the archive page will work...)

I've never personally seen anyone use voice recognition to write documents of any length or even to send texts (a few personal experiments with Siri notwithstanding).

I agree with your judgment of the prediction; however it's worth noting that the technology exists. The principal investigator of my former research group had problems with his hands caused by excessive typing, so he used voice recognition software to write journal papers.

Thanks! Corrected typos.

Another typo:

One the other hand...

Having been a TA at two universities in two different states, I can assure you that considering university students would increase the prevalence of small laptops & tablets, not decrease it. Although not literally true, it is perfectly true in the colloquial sense that everyone has them.

Restricting the sample to just the students I've taught (several hundred, probably less than a thousand), I'd view prediction 20 as mostly true in all but the most literal sense. (For instance, I find the difference between touch interfaces with fingers vs those with a stylus to be irrelevant to the spirit of overall truth of the prediction.)

For comparison, here is a self-assessment of predictions Scott Adams made in The Religion War, about 10 years ago.

Some of the predictions he mentions:

  • Islamic terrorists would regularly bomb targets in the United States using small “suicide” drones equipped with explosives and GPS guidance

  • there would be so much data available about individual behavior that skilled programmers could mine it to make freakishly accurate predictions.

The first is false, the second true. Hm, if Adams managed an overall 50% hit rate, then he's close to competitive with Kurzweil!

Apart from the belief that the animated personality would be visual, this is a near-perfect description of Siri and similar assistants. True.

It seems strange to call Siri ubiquitous when smartphone penetration among teenagers is less than 50%. (My own social circle seems to have below-average smartphone penetration, so I may not be well-calibrated.)

When you call 5 partially true and 20 partially false, which are you saying is more correct? 5 seems more correct to me, but "partially false" sounds more correct than "partially true".

Beyond musical recordings, images, and movie videos, the most popular type of digital entertainment object is virtual experience software.

Does this mean virtual experience software is more popular than the others, or that it's the most popular type of digital entertainment when you look beyond the others?

It seems strange to call Siri ubiquitous when smartphone penetration among teenagers is less than 50%

Ubiquitous is one of those words that are hard to quantify. "most people have one" is false; "most people know someone who has one" is true.

My scale would go: true, partially true, partially true and partially false, partially false, false.

Does this mean virtual experience software is more popular than the others, or that it's the most popular type of digital entertainment when you look beyond the others?

Both statements are wrong, at least according to the Economist, that put games just second to movies but above music. Again, ambiguity can't be used to make a false statement true...

It seems strange to call Siri ubiquitous when smartphone penetration among teenagers is less than 50%.

It also seems strange to call Siri ubiquitous when, on top of that, iOS only has (as of March 2012) between 30-45% market share (depending on how you measure it), which includes numerous models of iPhone that do not have/support Siri, as well as the numerous people who have access to, but don't primarily use Siri on their iPhones. (In my biased sample of software developer/cubicle dweller coworkers, as well as friends and family, I'd estimate maybe 5-10% of those who I know that have iPhones with Siri actually use Siri on a daily basis.)

Does this mean virtual experience software is more popular than the others, or that it's the most popular type of digital entertainment when you look beyond the others?

By my reading, the statement is saying that music, pictures and movies are more popular than "virtual experience software", and that VES is the next most popular.

Additionally, to respond to Stuart_Armstrong below, without a direct reference, I'd imagine that the Economist simply took into account popularity by sales data, which would ignore things like Pandora/Spotify/YouTube/Reddit usage/browsing that may happen significantly more than paid consumption of music/video (at least for certain segments of society with ubiquitous internet access).

Your judgment on all of these is ridiculously positive. Just about everything you claim as true or partly true seems to be mostly false to totally false to me.

I kind of agree. I ticked off the predictions in my own head before scrolling down to see everyone else's assessment, and here's what I decided. (I didn't consult Google or look things up, so take this with a pinch of AFAIK.)

  • 5: false. Wired mice, displays, and printers remain common, more common than their wireless equivalents in my experience. In absolute terms there are surely more wired computer components out there than in 1999.

  • 7: false. Even if the technology exists, I'm almost certain more text is still created by typing than CSR. And CSR is still less accurate than (sufficiently careful) human transcription; I vaguely remember Google recently beating Siri on this count.

  • 8: false. Even if one counts these LUIs as ubiquitous, they aren't frequently combined with animated personalities, and interacting with Siri et al. isn't much like talking to a person through video conferencing. I can't recall using LUIs or anything like them for simple business transactions (when I call businesses on the phone, for instance, it's usually a human, a recording, or a press-one-for-this-press-two-for-that menu that answers). Worse, calling my local cinema and navigating their non-CSR LUI shows that even when recognising simple phrases (like my town's name or a film's name) from a circumscribed list of possibilities some LUIs remain unresponsive and imprecise.

  • 18: weakly true. Computers aren't used in every classroom lesson and they're not in every classroom, but they're in almost every school and kids routinely use them for writing essays, learning through educational games, and doing research. Nowadays, they probably do learn more from school computers than home computers.

  • 20: false. Students now typically have a computer of their own but they aren't all smartphones. Those who do interact with smartphones don't mainly rely on styluses or speech and most of the text they enter is done with a keyboard (whether real or displayed virtually on-screen).

  • 26: false? The second sentence is true (as SA writes, screen readers confirm this prediction). However, I'm not aware of cheap, real-world, real-time, handheld OCR that reliably & automatically processes text on signs & displays (although I'm open to correction).

  • 29: true enough. I was going to call this false but that's just down to my own ignorance because I didn't know these systems existed. SA's NYT link shows they do.

  • 44: false. Intelligent roads, as far as I know, basically don't exist (at least not in a novel form that didn't exist in the '90s).

  • 48: false. I'd call the first sentence true. The second is probably false and the third is surely false, given the impact of the Great Recession. For unskilled people the economy was surely worse on average in 2009 & 2011 than it was in 1999.

  • 53: false. I generously interpreted "virtual experience software" as computer games in general. These do let users "experience fantasy environments with no counterpart in the physical world", but chances are there still isn't a game out there where you can have virtual sex with your favourite actor. Moreover, the fact that few games are virtual reality games suggests that the visual and auditory experience of VR remains uncompelling.

So I'd give Kurzweil 10-20% here, not 50+%. I think the main reason is that SA was prepared to give Kurzweil a "weakly true" if most of a prediction was solid, whereas I required every part of a prediction to be basically right. If I broke the predictions down into individual sentences and scored those sentences one by one, Kurzweil would score higher.

So I'd give Kurzweil 10-20% here, not 50+%. I think the main reason is that SA was prepared to give Kurzweil a "weakly true" if most of a prediction was solid, whereas I required every part of a prediction to be basically right. If I broke the predictions down into individual sentences and scored those sentences one by one, Kurzweil would score higher.

I'm very much of mind that scoring this way was the correct course, since the predictions are cunjunctive, and for any prediction to be strong evidence for his overarching claims all its parts must be coherent and true. Intuitively, applying Occam's Razor should, IMO, produce the results you've obtained.

This is far more sensible judgement of Kurzweil's prediction than OP's.

Interesting. I was convinced I was erring on the other side. Which is another indication of how bloody subjective assessing these predictions is.