Previously in series: The Nature of Logic

People who don't work in AI, who hear that I work in AI, often ask me: "Do you build neural networks or expert systems?" This is said in much the same tones as "Are you a good witch or a bad witch?"

Now that's what I call successful marketing.

Yesterday I covered what I see when I look at "logic" as an AI technique. I see something with a particular shape, a particular power, and a well-defined domain of useful application where cognition is concerned. Logic is good for leaping from crisp real-world events to compact general laws, and then verifying that a given manipulation of the laws preserves truth. It isn't even remotely close to the whole, or the center, of a mathematical outlook on cognition.

But for a long time, years and years, there was a tremendous focus in Artificial Intelligence on what I call "suggestively named LISP tokens" - a misuse of logic to try to handle cases like "Socrates is human, all humans are mortal, therefore Socrates is mortal". For many researchers, this one small element of math was indeed their universe.

And then along came the amazing revolution, the new AI, namely connectionism.

In the beginning (1957) was Rosenblatt's Perceptron. It was, I believe, billed as being inspired by the brain's biological neurons. The Perceptron had exactly two layers, a set of input units, and a single binary output unit. You multiplied the inputs by the weightings on those units, added up the results, and took the sign: that was the classification. To learn from the training data, you checked the current classification on an input, and if it was wrong, you dropped a delta on all the weights to nudge the classification in the right direction.

The Perceptron could only learn to deal with training data that was linearly separable - points in a hyperspace that could be cleanly separated by a hyperplane.

And that was all that this amazing algorithm, "inspired by the brain", could do.

In 1969, Marvin Minsky and Seymour Papert pointed out that Perceptrons couldn't learn the XOR function because it wasn't linearly separable. This killed off research in neural networks for the next ten years.

Now, you might think to yourself: "Hey, what if you had more than two layers in a neural network? Maybe then it could learn the XOR function?"

Well, b...

IgnoranceNeverPays

IgnoranceNeverPays

IgnoranceNeverPays

IgnoranceNeverPays