Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Formative Youth

20 Eliezer_Yudkowsky 24 February 2009 11:02PM

Followup toAgainst Maturity

"Rule of thumb:  Be skeptical of things you learned before you could read.  E.g., religion."
        -- Ben Casnocha

Looking down on others is fun, and if there's one group we adults can all enjoy looking down on, it's children.  At least I assume this is one of the driving forces behind the incredible disregard for... but don't get me started.

Inconveniently, though, most of us were children at one point or another during our lives.  Furthermore, many of us, as adults, still believe or choose certain things that we happened to believe or choose as children.  This fact is incongruent with the general fun of condescension - it means that your life is being run by a child, even if that particular child happens to be your own past self.

I suspect that most of us therefore underestimate the degree to which our youths were formative - because to admit that your youth was formative is to admit that the course of your life was not all steered by Incredibly Deep Wisdom and uncaused free will.

To give a concrete example, suppose you asked me, "Eliezer, where does your altruism originally come from?  What was the very first step in the chain that made you amenable to helping others?"

Then my best guess would be "Watching He-Man and similar TV shows as a very young and impressionable child, then failing to compartmentalize the way my contemporaries did."  (Same reason my Jewish education didn't take; I either genuinely believed something, or didn't believe it at all.  (Not that I'm saying that I believed He-Man was fact; just that the altruistic behavior I picked up wasn't compartmentalized off into some safely harmless area of my brain, then or later.))

It's my understanding that most people would be reluctant to admit this sort of historical fact, because it makes them sound childish - in the sense that they're still being governed by the causal history of a child.

But I find myself skeptical that others are governed by their childhood causal histories so much less than myself - especially when there's a simple alternative explanation: they're too embarrassed to admit it.

continue reading »

Pretending to be Wise

27 Eliezer_Yudkowsky 19 February 2009 10:30PM

Followup toAgainst Maturity

"The hottest place in Hell is reserved for those who in time of crisis remain neutral."
        -- Dante Alighieri, famous hell expert John F. Kennedy, misquoter

A special case of adulthood-signaling worth singling out, is the display of neutrality or suspended judgment, in order to signal maturity, wisdom, impartiality, or just a superior vantage point.

An example would be the case discussed yesterday of my parents, who respond to theological questions like "Why does ancient Egypt, which had good records on many other matters, lack any records of Jews having ever been there?" with "Oh, when I was your age, I also used to ask that sort of question, but now I've grown out of it."

Another example would be the principal who, faced with two children who were caught fighting on the playground, sternly says:  "It doesn't matter who started the fight, it only matters who ends it."  Of course it matters who started the fight.  The principal may not have access to good information about this critical fact, but if so, he should say so, not dismiss the importance of who threw the first punch.  Let a parent try punching the principal, and we'll see how far "It doesn't matter who started it" gets in front of a judge.  But to adults it is just inconvenient that children fight, and it matters not at all to their convenience which child started it, it is only convenient that the fight end as rapidly as possible.

A similar dynamic, I believe, governs the occasions in international diplomacy where Great Powers sternly tell smaller groups to stop that fighting right now.  It doesn't matter to the Great Power who started it - who provoked, or who responded disproportionately to provocation - because the Great Power's ongoing inconvenience is only a function of the ongoing conflict.  Oh, can't Israel and Hamas just get along?

This I call "pretending to be Wise".  Of course there are many ways to try and signal wisdom.  But trying to signal wisdom by refusing to make guesses - refusing to sum up evidence - refusing to pass judgment - refusing to take sides - staying above the fray and looking down with a lofty and condescending gaze - which is to say, signaling wisdom by saying and doing nothing - well, that I find particularly pretentious.

continue reading »

Against Maturity

28 Eliezer_Yudkowsky 18 February 2009 11:34PM

I remember the moment of my first break with Judaism.  It was in kindergarten, when I was being forced to memorize and recite my first prayer.  It was in Hebrew.  We were given a transliteration, but not a translation.  I asked what the prayer meant.  I was told that I didn't need to know - so long as I prayed in Hebrew, it would work even if I didn't understand the words.  (Any resemblance to follies inveighed against in my writings is not coincidental.)

Of course I didn't accept this, since it was blatantly stupid, and I figured that God had to be at least as smart as I was.  So when I got home, I asked my parents, and they didn't bother arguing with me.  They just said, "You're too young to argue with; we're older and wiser; adults know best; you'll understand when you're older."

They were right about that last part, anyway.

Of course there were plenty of places my parents really did know better, even in the realms of abstract reasoning.  They were doctorate-bearing folks and not stupid.  I remember, at age nine or something silly like that, showing my father a diagram full of filled circles and trying to convince him that the indeterminacy of particle collisions was because they had a fourth-dimensional cross-section and they were bumping or failing to bump in the fourth dimension.

My father shot me down flat.  (Without making the slightest effort to humor me or encourage me.  This seems to have worked out just fine.  He did buy me books, though.)

But he didn't just say, "You'll understand when you're older."  He said that physics was math and couldn't even be talked about without math.  He talked about how everyone he met tried to invent their own theory of physics and how annoying this was.  He may even have talked about the futility of "providing a mechanism", though I'm not actually sure if I originally got that off him or Baez.

You see the pattern developing here.  "Adulthood" was what my parents appealed to when they couldn't verbalize any object-level justification.  They had doctorates and were smart; if there was a good reason, they usually would at least try to explain it to me.  And it gets worse...

continue reading »

The Level Above Mine

44 Eliezer_Yudkowsky 26 September 2008 09:18AM

Followup toThe Proper Use of Humility, Tsuyoku Naritai

(At this point, I fear that I must recurse into a subsequence; but if all goes as planned, it really will be short.)

I once lent Xiaoguang "Mike" Li my copy of "Probability Theory: The Logic of Science".  Mike Li read some of it, and then came back and said:

"Wow... it's like Jaynes is a thousand-year-old vampire."

Then Mike said, "No, wait, let me explain that—" and I said, "No, I know exactly what you mean."  It's a convention in fantasy literature that the older a vampire gets, the more powerful they become.

I'd enjoyed math proofs before I encountered Jaynes.  But E.T. Jaynes was the first time I picked up a sense of formidability from mathematical arguments.  Maybe because Jaynes was lining up "paradoxes" that had been used to object to Bayesianism, and then blasting them to pieces with overwhelming firepower—power being used to overcome others.  Or maybe the sense of formidability came from Jaynes not treating his math as a game of aesthetics; Jaynes cared about probability theory, it was bound up with other considerations that mattered, to him and to me too.

For whatever reason, the sense I get of Jaynes is one of terrifying swift perfection—something that would arrive at the correct answer by the shortest possible route, tearing all surrounding mistakes to shreds in the same motion.  Of course, when you write a book, you get a chance to show only your best side.  But still.

continue reading »

A Prodigy of Refutation

19 Eliezer_Yudkowsky 18 September 2008 01:57AM

Followup toMy Childhood Death Spiral, Raised in Technophilia

My Childhood Death Spiral described the core momentum carrying me into my mistake, an affective death spiral around something that Eliezer1996 called "intelligence".  I was also a technophile, pre-allergized against fearing the future.  And I'd read a lot of science fiction built around personhood ethics—in which fear of the Alien puts humanity-at-large in the position of the bad guys, mistreating aliens or sentient AIs because they "aren't human".

That's part of the ethos you acquire from science fiction—to define your in-group, your tribe, appropriately broadly.  Hence my email address, sentience@pobox.com.

So Eliezer1996 is out to build superintelligence, for the good of humanity and all sentient life.

At first, I think, the question of whether a superintelligence will/could be good/evil didn't really occur to me as a separate topic of discussion.  Just the standard intuition of, "Surely no supermind would be stupid enough to turn the galaxy into paperclips; surely, being so intelligent, it will also know what's right far better than a human being could."

Until I introduced myself and my quest to a transhumanist mailing list, and got back responses along the general lines of (from memory):

continue reading »

Raised in Technophilia

20 Eliezer_Yudkowsky 17 September 2008 02:06AM

Followup toMy Best and Worst Mistake

My father used to say that if the present system had been in place a hundred years ago, automobiles would have been outlawed to protect the saddle industry.

One of my major childhood influences was reading Jerry Pournelle's A Step Farther Out, at the age of nine.  It was Pournelle's reply to Paul Ehrlich and the Club of Rome, who were saying, in the 1960s and 1970s, that the Earth was running out of resources and massive famines were only years away.  It was a reply to Jeremy Rifkin's so-called fourth law of thermodynamics; it was a reply to all the people scared of nuclear power and trying to regulate it into oblivion.

I grew up in a world where the lines of demarcation between the Good Guys and the Bad Guys were pretty clear; not an apocalyptic final battle, but a battle that had to be fought over and over again, a battle where you could see the historical echoes going back to the Industrial Revolution, and where you could assemble the historical evidence about the actual outcomes.

On one side were the scientists and engineers who'd driven all the standard-of-living increases since the Dark Ages, whose work supported luxuries like democracy, an educated populace, a middle class, the outlawing of slavery.

On the other side, those who had once opposed smallpox vaccinations, anesthetics during childbirth, steam engines, and heliocentrism:  The theologians calling for a return to a perfect age that never existed, the elderly white male politicians set in their ways, the special interest groups who stood to lose, and the many to whom science was a closed book, fearing what they couldn't understand.

And trying to play the middle, the pretenders to Deep Wisdom, uttering cached thoughts about how technology benefits humanity but only when it was properly regulated—claiming in defiance of brute historical fact that science of itself was neither good nor evil—setting up solemn-looking bureaucratic committees to make an ostentatious display of their caution—and waiting for their applause.  As if the truth were always a compromise.  And as if anyone could really see that far ahead.  Would humanity have done better if there'd been a sincere, concerned, public debate on the adoption of fire, and commitees set up to oversee its use?

continue reading »

My Best and Worst Mistake

22 Eliezer_Yudkowsky 16 September 2008 12:43AM

Followup toMy Childhood Death Spiral

Yesterday I covered the young Eliezer's affective death spiral around something that he called "intelligence".  Eliezer1996, or even Eliezer1999 for that matter, would have refused to try and put a mathematical definition—consciously, deliberately refused.  Indeed, he would have been loath to put any definition on "intelligence" at all.

Why?  Because there's a standard bait-and-switch problem in AI, wherein you define "intelligence" to mean something like "logical reasoning" or "the ability to withdraw conclusions when they are no longer appropriate", and then you build a cheap theorem-prover or an ad-hoc nonmonotonic reasoner, and then say, "Lo, I have implemented intelligence!"  People came up with poor definitions of intelligence—focusing on correlates rather than cores—and then they chased the surface definition they had written down, forgetting about, you know, actual intelligence.  It's not like Eliezer1996 was out to build a career in Artificial Intelligence.  He just wanted a mind that would actually be able to build nanotechnology.  So he wasn't tempted to redefine intelligence for the sake of puffing up a paper.

Looking back, it seems to me that quite a lot of my mistakes can be defined in terms of being pushed too far in the other direction by seeing someone else stupidity:  Having seen attempts to define "intelligence" abused so often, I refused to define it at all.  What if I said that intelligence was X, and it wasn't really X?  I knew in an intuitive sense what I was looking for—something powerful enough to take stars apart for raw material—and I didn't want to fall into the trap of being distracted from that by definitions.

Similarly, having seen so many AI projects brought down by physics envy—trying to stick with simple and elegant math, and being constrained to toy systems as a result—I generalized that any math simple enough to be formalized in a neat equation was probably not going to work for, you know, real intelligence.  "Except for Bayes's Theorem," Eliezer2000 added; which, depending on your viewpoint, either mitigates the totality of his offense, or shows that he should have suspected the entire generalization instead of trying to add a single exception.

continue reading »

My Childhood Death Spiral

23 Eliezer_Yudkowsky 15 September 2008 03:42AM

Followup toAffective Death Spirals, My Wild and Reckless Youth

My parents always used to downplay the value of intelligence.  And play up the value of—effort, as recommended by the latest research?  No, not effort.  Experience.  A nicely unattainable hammer with which to smack down a bright young child, to be sure.  That was what my parents told me when I questioned the Jewish religion, for example.  I tried laying out an argument, and I was told something along the lines of:  "Logic has limits, you'll understand when you're older that experience is the important thing, and then you'll see the truth of Judaism."  I didn't try again.  I made one attempt to question Judaism in school, got slapped down, didn't try again.  I've never been a slow learner.

Whenever my parents were doing something ill-advised, it was always, "We know better because we have more experience.  You'll understand when you're older: maturity and wisdom is more important than intelligence."

If this was an attempt to focus the young Eliezer on intelligence uber alles, it was the most wildly successful example of reverse psychology I've ever heard of.

But my parents aren't that cunning, and the results weren't exactly positive.

continue reading »

View more: Next