Followup to:  Guessing the Teacher's Password, Artificial Addition

A classic paper by Drew McDermott, "Artificial Intelligence Meets Natural Stupidity", criticized AI programs that would try to represent notions like happiness is a state of mind using a semantic network:

STATE-OF-MIND
     ^
     | IS-A
     |
HAPPINESS

And of course there's nothing inside the "HAPPINESS" node; it's just a naked LISP token with a suggestive English name.

So, McDermott says, "A good test for the disciplined programmer is to try using gensyms in key places and see if he still admires his system.  For example, if STATE-OF-MIND is renamed G1073..." then we would have IS-A(HAPPINESS, G1073) "which looks much more dubious."

Or as I would slightly rephrase the idea:  If you substituted randomized symbols for all the suggestive English names, you would be completely unable to figure out what G1071(G1072, 1073) meant.  Was the AI program meant to represent hamburgers?  Apples?  Happiness?  Who knows?  If you delete the suggestive English names, they don't grow back.

Followup to:  Semantic Stopsigns, We Don't Really Want Your Participation

At the Singularity Summit 2007, one of the speakers called for democratic, multinational development of AI.  So I stepped up to the microphone and asked:

Suppose that a group of democratic republics form a consortium to develop AI, and there's a lot of politicking during the process—some interest groups have unusually large influence, others get shafted—in other words, the result looks just like the products of modern democracies.  Alternatively, suppose a group of rebel nerds develops an AI in their basement, and instructs the AI to poll everyone in the world—dropping cellphones to anyone who doesn't have them—and do whatever the majority says.  Which of these do you think is more "democratic", and would you feel safe with either?

I wanted to find out whether he believed in the pragmatic adequacy of the democratic political process, or if he believed in the moral rightness of voting.  But the speaker replied:

The first scenario sounds like an editorial in Reason magazine, and the second sounds like a Hollywood movie plot.

Confused, I asked:

Then what kind of democratic process did you have in mind?

Followup to: Semantic Stopsigns, Mysterious Answers to Mysterious Questions, Say Not 'Complexity'

Imagine that I, in full view of live television cameras, raised my hands and chanted abracadabra and caused a brilliant light to be born, flaring in empty space beyond my outstretched hands.  Imagine that I committed this act of blatant, unmistakeable sorcery under the full supervision of James Randi and all skeptical armies. Most people, I think, would be fairly curious as to what was going on.

But now suppose instead that I don't go on television.  I do not wish to share the power, nor the truth behind it. I want to keep my sorcery secret.  And yet I also want to cast my spells whenever and wherever I please. I want to cast my brilliant flare of light so that I can read a book on the train—without anyone becoming curious.  Is there a spell that stops curiosity?

Yes indeed!  Whenever anyone asks "How did you do that?", I just say "Science!"

It's not a real explanation, so much as a curiosity-stopper. It doesn't tell you whether the light will brighten or fade, change color in hue or saturation, and it certainly doesn't tell you how to make a similar light yourself. You don't actually know anything more than you knew before I said the magic word. But you turn away, satisfied that nothing unusual is going on.

Better yet, the same trick works with a standard light switch.

Followup to: Semantic Stopsigns, Mysterious Answers to Mysterious Questions

As our tribe wanders through the grasslands, searching for fruit trees and prey, it happens every now and then that water pours down from the sky.

"Why does water sometimes fall from the sky?" I ask the bearded wise man of our tribe.

He thinks for a moment, this question having never occurred to him before, and then says, "From time to time, the sky spirits battle, and when they do, their blood drips from the sky."

"Where do the sky spirits come from?" I ask.

His voice drops to a whisper.  "From the before time.  From the long long ago."

When it rains, and you don't know why, you have several options.  First, you could simply not ask why—not follow up on the question, or never think of the question in the first place.  This is the Ignore command, which the bearded wise man originally selected.  Second, you could try to devise some sort of explanation, the Explain command, as the bearded man did in response to your first question.  Third, you could enjoy the sensation of mysteriousness—the Worship command.

Followup to:  Failing to Learn from History

There is a habit of thought which I call the logical fallacy of generalization from fictional evidence, which deserves a blog post in its own right, one of these days.  Journalists who, for example, talk about the Terminator movies in a report on AI, do not usually treat Terminator as a prophecy or fixed truth.  But the movie is recalled—is available—as if it were an illustrative historical case.  As if the journalist had seen it happen on some other planet, so that it might well happen here.  More on this in Section 6 of this paper.

There is an inverse error to generalizing from fictional evidence: failing to be sufficiently moved by historical evidence.  The trouble with generalizing from fictional evidence is that it is fiction—it never actually happened.  It's not drawn from the same distribution as this, our real universe; fiction differs from reality in systematic ways.  But history has happened, and should be available.

In our ancestral environment, there were no movies; what you saw with your own eyes was true.  Is it any wonder that fictions we see in lifelike moving pictures have too great an impact on us?  Conversely, things that really happened, we encounter as ink on paper; they happened, but we never saw them happen.  We don't remember them happening to us.

It is said that parents do all the things they tell their children not to do, which is how they know not to do them.

Long ago, in the unthinkably distant past, I was a devoted Traditional Rationalist, conceiving myself skilled according to that kind, yet I knew not the Way of Bayes.  When the young Eliezer was confronted with a mysterious-seeming question, the precepts of Traditional Rationality did not stop him from devising a Mysterious Answer.  It is, by far, the most embarrassing mistake I made in my life, and I still wince to think of it.

What was my mysterious answer to a mysterious question?  This I will not describe, for it would be a long tale and complicated.  I was young, and a mere Traditional Rationalist who knew not the teachings of Tversky and Kahneman.  I knew about Occam's Razor, but not the conjunction fallacy.  I thought I could get away with thinking complicated thoughts myself, in the literary style of the complicated thoughts I read in science books, not realizing that correct complexity is only possible when every step is pinned down overwhelmingly.  Today, one of the chief pieces of advice I give to aspiring young rationalists is "Do not attempt long chains of reasoning or complicated plans."

Nothing more than this need be said:  Even after I invented my "answer", the phenomenon was still a mystery unto me, and possessed the same quality of wondrous impenetrability that it had at the start.

Make no mistake, that younger Eliezer was not stupid.  All the errors of which the young Eliezer was guilty, are still being made today by respected scientists in respected journals.  It would have taken a subtler skill to protect him, than ever he was taught as a Traditional Rationalist.

Indeed, the young Eliezer diligently and painstakingly followed the injunctions of Traditional Rationality in the course of going astray.

Once upon a time...

This is a story from when I first met Marcello, with whom I would later work for a year on AI theory; but at this point I had not yet accepted him as my apprentice.  I knew that he competed at the national level in mathematical and computing olympiads, which sufficed to attract my attention for a closer look; but I didn't know yet if he could learn to think about AI.

I had asked Marcello to say how he thought an AI might discover how to solve a Rubik's Cube.  Not in a preprogrammed way, which is trivial, but rather how the AI itself might figure out the laws of the Rubik universe and reason out how to exploit them.  How would an AI invent for itself the concept of an "operator", or "macro", which is the key to solving the Rubik's Cube?

At some point in this discussion, Marcello said:  "Well, I think the AI needs complexity to do X, and complexity to do Y—"

And I said, "Don't say 'complexity'."

Marcello said, "Why not?"

And the child asked:

Q:  Where did this rock come from?
A:  I chipped it off the big boulder, at the center of the village.
Q:  Where did the boulder come from?
A:  It probably rolled off the huge mountain that towers over our village.
Q:  Where did the mountain come from?
A:  The same place as all stone: it is the bones of Ymir, the primordial giant.
Q:  Where did the primordial giant, Ymir, come from?
A:  From the great abyss, Ginnungagap.
Q:  Where did the great abyss, Ginnungagap, come from?
A:  Never ask that question.

Consider the seeming paradox of the First Cause.  Science has traced events back to the Big Bang, but why did the Big Bang happen?  It's all well and good to say that the zero of time begins at the Big Bang—that there is nothing before the Big Bang in the ordinary flow of minutes and hours.  But saying this presumes our physical law, which itself appears highly structured; it calls out for explanation.  Where did the physical laws come from?  You could say that we're all a computer simulation, but then the computer simulation is running on some other world's laws of physics—where did those laws of physics come from?

At this point, some people say, "God!"

Followup to:  Universal Fire

Antoine-Laurent de Lavoisier discovered that breathing (respiration) and fire (combustion) operated on the same principle.  It was one of the most startling unifications in the history of science, for it brought together the mundane realm of matter and the sacred realm of life, which humans had divided into separate magisteria.

The first great simplification was that of Isaac Newton, who unified the course of the planets with the trajectory of a falling apple.  The shock of this discovery was greater by far than Lavoisier's.  It wasn't just that Newton had dared to unify the Earthly realm of base matter with the obviously different and sacred celestial realm, once thought to be the abode of the gods.  Newton's discovery gave rise to the notion of a universal law, one that is the same everywhere and everywhen, with literally zero exceptions.

In L. Sprague de Camp's fantasy story The Incomplete Enchanter (which set the mold for the many imitations that followed), the hero, Harold Shea, is transported from our own universe into the universe of Norse mythology.  This world is based on magic rather than technology; so naturally, when Our Hero tries to light a fire with a match brought along from Earth, the match fails to strike.

I realize it was only a fantasy story, but... how do I put this...

No.