Followup to: The Level Above Mine, Competent Elites

(Those who didn't like the last two posts should definitely skip this one.)

I recall one fellow, who seemed like a nice person, and who was quite eager to get started on Friendly AI work, to whom I had trouble explaining that he didn't have a hope. He said to me:

"If someone with a Masters in chemistry isn't intelligent enough, then you're not going to have much luck finding someone to help you."

It's hard to distinguish the grades above your own. And even if you're literally the best in the world, there are still electron orbitals above yours—they're just unoccupied. Someone had to be "the best physicist in the world" during the time of Ancient Greece. Would they have been able to visualize Newton?

At one of the first conferences organized around the tiny little subfield of Artificial General Intelligence, I met someone who was heading up a funded research project specifically declaring AGI as a goal, within a major corporation. I believe he had people under him on his project. He was probably paid at least three times as much as I was paid (at that time). His academic credentials were superior to mine (what a surprise) and he had many more years of experience. He had access to lots and lots of computing power.

And like nearly everyone in the field of AGI, he was rushing forward to write code immediately—not holding off and searching for a sufficiently precise theory to permit stable self-improvement.

In short, he was just the sort of fellow that... Well, many people, when they hear about Friendly AI, say: "Oh, it doesn't matter what you do, because [someone like this guy] will create AI first." He's the sort of person about whom journalists ask me, "You say that this isn't the time to be talking about regulation, but don't we need laws to stop people like this from creating AI?"

"I suppose," you say, your voice heavy with irony, "that you're about to tell us, that this person doesn't really have so much of an advantage over you as it might seem. Because your theory—whenever you actually come up with a theory—is going to be so much better than his. Or," your voice becoming even more ironic, "that he's too mired in boring mainstream methodology—"

No. I'm about to tell you that I happened to be ...

Tim_O'Brien

Tim_O'Brien

Tim_O'Brien

Tim_O'Brien