I wouldn't call Google's search personalization "self-training" because the user is responsible for adding new data points to his or her own model; it's the same online algorithm it's always been, just tailored to billions of individual users rather than a set of billions of users. The set of links that a user has clicked on through Google searches is updated every time the user clicks a new link, and the algorithm uses this to tweak the ordering of presented search results, but AFAIK the algorithm has no way to evaluate whether the model update actually brought the ordering closer to the user's preferred ordering unless the user tells it so by clicking on one of the results. It could compare the ordering it did present to the ordering it would have presented if some set of data points wasn't in the model, but then it would have to have some heuristic for which points to drop for cross-validation.
The way I see it, the difference between an online algorithm and a self-training AI is that the latter would not only need such a heuristic -- let's call it "knowledge base evaluation" -- it would also need to be able to evaluate the fitness of novel knowledge base evaluation heuristics. (I'm torn as to whether that goalpost should also include "can generate novel KBE heuristics"; I'll have to think about that a while longer.) Even so, as long as the user dictates which points the algorithm can even consider adding to its KB, the user is acting as a gatekeeper on what knowledge the algorithm can acquire.
The way I see it, the difference between an online algorithm and a self-training AI is that the latter would not only need such a heuristic -- let's call it "knowledge base evaluation" -- it would also need to be able to evaluate the fitness of novel knowledge base evaluation heuristics.
On reflection, I'm now contradicting my original statement; the above is a stab toward an algorithmic notion of "self-training" that is orthogonal to how restricted an algorithm's training input set is, or who is restricting it, or how. Using this hal...
Thinking aloud:
Humans are examples of general intelligence - the only example we're sure of. Some humans have various degrees of autism (low level versions are quite common in the circles I've moved in), impairing their social skills. Mild autists nevertheless remain general intelligences, capable of demonstrating strong cross domain optimisation. Psychology is full of other examples of mental pathologies that impair certain skills, but nevertheless leave their sufferers as full fledged general intelligences. This general intelligence is not enough, however, to solve their impairments.
Watson triumphed on Jeopardy. AI scientists in previous decades would have concluded that to do so, a general intelligence would have been needed. But that was not the case at all - Watson is blatantly not a general intelligence. Big data and clever algorithms were all that were needed. Computers are demonstrating more and more skills, besting humans in more and more domains - but still no sign of general intelligence. I've recently developed the suspicion that the Turing test (comparing AI with a standard human) could get passed by a narrow AI finely tuned to that task.
The general thread is that the link between narrow skills and general intelligence may not be as clear as we sometimes think. It may be that narrow skills are sufficiently diverse and unique that a mid-level general intelligence may not be able to develop them to a large extent. Or, put another way, an above-human social intelligence may not be able to control a robot body or do decent image recognition. A super-intelligence likely could: ultimately, general intelligence includes the specific skills. But his "ultimately" may take a long time to come.
So the questions I'm wondering about are: