Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Final Words

71 Post author: Eliezer_Yudkowsky 27 April 2009 09:12PM

Sunlight enriched air already alive with curiosity, as dawn rose on Brennan and his fellow students in the place to which Jeffreyssai had summoned them.

They sat there and waited, the five, at the top of the great glassy crag that was sometimes called Mount Mirror, and more often simply left unnamed.  The high top and peak of the mountain, from which you could see all the lands below and seas beyond.

(Well, not all the lands below, nor seas beyond.  So far as anyone knew, there was no place in the world from which all the world was visible; nor, equivalently, any kind of vision that would see through all obstacle-horizons.  In the end it was the top only of one particular mountain: there were other peaks, and from their tops you would see other lands below; even though, in the end, it was all a single world.)

"What do you think comes next?" said Hiriwa.  Her eyes were bright, and she gazed to the far horizons like a lord.

Taji shrugged, though his own eyes were alive with anticipation.  "Jeffreyssai's last lesson doesn't have any obvious sequel that I can think of.  In fact, I think we've learned just about everything that I knew the beisutsukai masters know.  What's left, then -"

"Are the real secrets," Yin completed the thought.

Hiriwa and Taji and Yin shared a grin, among themselves.

Styrlyn wasn't smiling.  Brennan suspected rather strongly that Styrlyn was older than he had admitted.

Brennan wasn't smiling either.  He might be young, but he kept high company, and had witnesssed some of what went on behind the curtains of the world.  Secrets had their price, always, that was the barrier that made them secrets; and Brennan thought he had a good idea of what this price might be.

There was a cough from behind them, at a moment when they had all happened to be looking in any other direction but that one.

As one, their heads turned.

Jeffreyssai stood there, in a casual robe that looked more like glass than any proper sort of mirrorweave.

Jeffreyssai stood there and looked at them, a strange abiding sorrow in those inscrutable eyes.

"Sen...sei," Taji started, faltering as that bright anticipation stumbled over Jeffreyssai's return look.  "What's next?"

"Nothing," Jeffreyssai said abruptly.  "You're finished.  It's done."

Hiriwa, Taji, and Yin all blinked, a perfect synchronized gesture of shock.  Then, before their expressions could turn to outrage and objections -

"Don't," Jeffreyssai said.  There was real pain in it.  "Believe me, it hurts me more than it hurts you."  He might have been looking at them; or at something far away, or long ago.  "I don't know exactly what roads may lie before you - but yes, I know that you're not ready.  That I'm sending you out unprepared.  That everything I taught you is incomplete.  I know that what I said is not what you heard.  That I left out the one most important thing.  That the rhythm at the center of everything is missing and astray.  I know that you will harm yourself in the course of trying to use what I taught.  So that I, personally, will have shaped, in some fashion unknown to me, the very knife that will cut you..."

"...that's the hell of being a teacher, you see," Jeffreyssai said.  Something grim flickered in his expression.  "Nonetheless, you're done.  Finished, for now.  What lies between you and mastery is not another classroom.  We are fortunate, or perhaps not fortunate, that the road to power does not wend only through lecture halls.  Else the quest would be boring to the bitter end.  Still, I cannot teach you; and so it is a moot point whether I would if I could.  There is no master here whose art is entirely inherited.  Even the beisutsukai have never discovered how to teach certain things; it is possible that such an event has been prohibited.  And so you can only arrive at mastery by using to the fullest the techniques you have already learned, facing challenges and apprehending them, mastering the tools you have been taught until they shatter in your hands -"

Jeffreyssai's eyes were hard, as though steeled in acceptance of unwelcome news.

"- and you are left in the midst of wreckage absolute.  That is where I, your teacher, am sending you.  You are not beisutsukai masters.  I cannot create masters.  I have never known how to create masters.  Go forth, then, and fail."

"But -" said Yin, and stopped herself.

"Speak," said Jeffreyssai.

"But then why," she said, "why teach us anything in the first place?"

Brennan's eyelids flickered some tiny amount.

It was enough for Jeffreyssai.  "Answer her, Brennan, if you think you know."

"Because," Brennan said, "if we were not taught, there would be no chance at all of our becoming masters."

"Even so," said Jeffreyssai.  "If you were not taught - then when you failed, you might simply think you had reached the limits of Reason itself.  You would be discouraged and bitter within your disaster.  You might not even realize when you had failed.  No; you have been shaped into something that may emerge from the wreckage, determined to remake your Art.  And then you may remember much that will help you.  I cannot create masters, but if you had not been taught, your chances would be - less."  His gaze passed over the group.  "It should be obvious, but understand that you cannot provoke the moment of your crisis artificially.  To teach you something, the catastrophe must come to you as a surprise. You must go as far as you can, as best you can, and fail honestly.  The higher road begins after the Art seems to fail you; though the reality will be that it was you who failed your Art."

Brennan made the gesture with his hand that indicated a question; and Jeffreyssai nodded in reply.

"Is this the only way in which Bayesian masters come to be, sensei?"

"I do not know," said Jeffreyssai, from which the overall state of the evidence was obvious enough.  "But I doubt there would ever be a road to mastery that goes only through the monastery.  We are the heirs in this world of mystics as well as scientists, just as the Competitive Conspiracy inherits from chessplayers alongside cagefighters.  We have turned our impulses to more constructive uses - but we must still stay on our guard against old failure modes."

Jeffreyssai took a breath.  "Three flaws above all are common among the beisutsukai.  The first flaw is to look just the slightest bit harder for flaws in arguments whose conclusions you would rather not accept.  If you cannot contain this aspect of yourself then every flaw you know how to detect will make you that much stupider.  This is the challenge which determines whether you possess the art or its opposite:  Intelligence, to be useful, must be used for something other than defeating itself."

"The second flaw is cleverness.  To invent great complicated plans and great complicated theories and great complicated arguments - or even, perhaps, plans and theories and arguments which are commended too much by their elegance and too little by their realism.  There is a widespread saying which runs:  'The vulnerability of the beisutsukai is well-known; they are prone to be too clever.'  Your enemies will know this saying, if they know you for a beisutsukai, so you had best remember it also.  And you may think to yourself:  'But if I could never try anything clever or elegant, would my life even be worth living?'  This is why cleverness is still our chief vulnerability even after its being well-known, like offering a Competitor a challenge that seems fair, or tempting a Bard with drama."

"The third flaw is underconfidence, though it will seem to you like modesty or humility.  You have learned so many flaws in your own nature, some of them impossible to fix, that you may think that the rule of wisdom is to confess your own inability.  You may question yourself, without resolution or testing to determine the self-answers.  You may refuse to decide, pending further evidence, when a quick decision is necessary.  You may take advice you should not take.  Jaded cynicism and sage despair are less fashionable than once they were, but you may still be tempted by them.  Or you may simply - lose momentum."

Jeffreyssai fell silent then.

He looked from each of them, one to the other, with quiet intensity.

And said at last, "Those are my final words to you.  If and when we meet next, you and I - if and when you return to this place, Brennan, or Hiriwa, or Taji, or Yin, or Styrlyn - I will no longer be your teacher."

And Jeffreyssai turned and walked swiftly away, heading back toward the glassy tunnel that had emitted him.

Even Brennan was shocked.  For a moment they were all speechless.

Then -

"Wait!" cried Hiriwa.  "What about our final words to you?  I never said -"

"I will tell you what my sensei told me," Jeffreyssai's voice came back as he disappeared.  "You can thank me after you return, if you return.  One of you at least seems likely to come back."

"No, wait, I -"  Hiriwa fell silent.  In the mirrored tunnel, the fractured reflections of Jeffreyssai were already fading.  She shook her head.  "Never... mind, then."

There was a brief, uncomfortable silence, as the five of them looked at each other.

"Good heavens," Taji said finally.  "Even the Bardic Conspiracy wouldn't try for that much drama."

Yin suddenly laughed.  "Oh, this was nothing.  You should have seen my send-off when I left Diamond Sea University."  She smiled.  "I'll tell you about it sometime - if you're interested."

Taji coughed.  "I suppose I should go back and... pack my things..."

"I'm already packed," Brennan said.  He smiled, ever so slightly, when the other three turned to look at him.

"Really?" Taji asked.  "What was the clue?"

Brennan shrugged with artful carelessness.  "Beyond a certain point, it is futile to inquire how a beisutsukai master knows a thing -"

"Come off it!" Yin said.  "You're not a beisutsukai master yet."

"Neither is Styrlyn," Brennan said.  "But he has already packed as well."  He made it a statement rather than a question, betting double or nothing on his image of inscrutable foreknowledge.

Styrlyn cleared his throat.  "As you say.  Other commitments call me, and I have already tarried longer than I planned.  Though, Brennan, I do feel that you and I have certain mutual interests, which I would be happy to discuss with you -"

"Styrlyn, my most excellent friend, I shall be happy to speak with you on any topic you desire," Brennan said politely and noncommitally, "if we should meet again."  As in, not now.  He certainly wasn't selling out his Mistress this early in their relationship.

There was an exchange of goodbyes, and of hints and offers.

And then Brennan was walking down the road that led toward or away from Mount Mirror (for every road is a two-edged sword), the glassy pebbles clicking under his feet.

He strode out along the path with purpose, vigor, and determination, just in case someone was watching.

Some time later he stopped, stepped off the path, and moved just far enough away to prevent anyone from finding him unless they were deliberately following.

Then Brennan sagged back against a tree-trunk.  It was a sparse clearing, with only a few trees poking out of the ground; not much present in the way of distracting scenery, unless you counted the red-tinted stream flowing out of a dark cave-mouth.  And Brennan deliberately faced away from that, leaving only the far grey of the horizons, and the blue sky and bright sun.

Now what?

He had thought that the Bayesian Conspiracy, of all the possible trainings that existed in this world, would have cleared up his uncertainty about what to do with the rest of his life.

Power, he'd sought at first.  Strength to prevent a repetition of the past.  "If you don't know what you need, take power" - so went the proverb.  He had gone first to the Competitive Conspiracy, then to the beisutsukai.

And now...

Now he felt more lost than ever.

He could think of things that made him happy, but nothing that he really wanted.

The passionate intensity that he'd come to associate with his Mistress, or with Jeffreyssai, or the other figures of power that he'd met... a life of pursuing small pleasures seemed to pale in comparison, next to that.

In a city not far from the center of the world, his Mistress waited for him (in all probability, assuming she hadn't gotten bored with her life and run away).  But to merely return, and then drift aimlessly, waiting to fall into someone else's web of intrigue... no.  That didn't seem like enough.

Brennan plucked a blade of grass from the ground and stared at it, half-unconsciously looking for anything interesting about it; an old, old game that his very first teacher had taught him, what now seemed like ages ago.

Why did I believe that going to Mount Mirror would tell me what I wanted?

Well, decision theory did require that your utility function be consistent, but...

If the beisutsukai knew what I wanted, would they even tell me?

At Mount Mirror they taught doubt.  So now he was falling prey to the third besetting sin of which Jeffreyssai had spoken, lost momentum, for he had learned to question the image that he held of himself in his mind.

Are you seeking power because that is your true desire, Brennan?

Or because you have a picture in your mind, of the role that you play as an ambitious young man, and you think it is what someone playing your role would do?

Almost everything he'd done up until now, even going to Mount Mirror, had probably been the latter.

And when he blanked out the old thoughts and tried to see the problem as though for the first time...

...nothing much came to mind.

What do I want?

Maybe it wasn't reasonable to expect the beisutsukai to tell him outright.  But was there anything they had taught him by which he might answer?

Brennan closed his eyes and thought.

First, suppose there is something I would passionately desire.  Why would I not know what it is?

Because I have not yet encountered it, or ever imagined it?

Or because there is some reason I would not admit it to myself?

Brennan laughed out loud, then, and opened his eyes.

So simple, once you thought of it that way.  So obvious in retrospect.  That was what they called a silver-shoes moment, and yet, if he hadn't gone to Mount Mirror, it wouldn't ever have occurred to him.

Of course there was something he wanted.  He knew exactly what he wanted.  Wanted so desperately he could taste it like an sharp tinge on his tongue.

It just hadn't come to mind earlier, because... if he acknowledged his desire explicitly... then he also had to see that it was difficult.  High, high, above him.  Far out of his reach.  "Impossible" was the word that came to mind, though it was not, of course, physically impossible.

But once he asked himself if he preferred to wander aimlessly through his life - once it was put that way, the answer became obvious.  Pursuing the unattainable would make for a hard life, but not a sad one.  He could think of things that made him happy, either way.  And in the end - it was what he wanted.

Brennan stood up, and took his first steps, in the exact direction of Shir L'or, the city that lies in the center of the world.  He had a plot to hatch, and he did not know who would be part of it.

And then Brennan almost stumbled, when he realized that Jeffreyssai had already known.

One of you at least seems likely to come back...

Brennan had thought he was talking about Taji.  Taji had probably thought he was talking about Taji.  It was what Taji said he wanted.  But how reliable of an indicator was that, really?

There was a proverb about that very road he had just left:  Whoever sets out from Mount Mirror seeking the impossible, will surely return.

When you considered Jeffreyssai's last warning - and that the proverb said nothing of succeeding at the impossible task itself - it was a less optimistic saying than it sounded.

Brennan shook his head wonderingly.  How could Jeffreyssai possibly have known before Brennan knew himself?

Well, beyond a certain point, it is futile to inquire how a beisutsukai master knows a thing -

Brennan halted in mid-thought.

No.

No, if he was going to become a beisutsukai master himself someday, then he ought to figure it out.

It was, Brennan realized, a stupid proverb.

So he walked, and this time, he thought about it carefully.

As the sun was setting, red-golden, shading his footsteps in light.

Comments (54)

Comment author: Kaj_Sotala 28 April 2009 04:19:43PM *  15 points [-]

Beautiful.

And it almost makes me want to be Brennan - with all the training of the Bayesian Conspiracy could give me, a fiercely desired goal in my mind, and the whole world open in front of me. (And - presumably - no existential risks to worry about.)

Comment author: Baughn 15 May 2009 12:31:20AM 7 points [-]

You're assuming that his goal is not, in itself, an existential risk. Something that is almost, but not quite, unattainable.. we know of such a technology already, and Eliezer is aiming for it.

Comment author: cousin_it 27 April 2009 10:59:42PM *  10 points [-]

Of course there was something he wanted. He knew exactly what he wanted. Wanted so desperately he could taste it like an sharp tinge on his tongue.

This alone would've been the most useful rationalist technique ever if it worked. But it doesn't work. Wanting to want doesn't make me want.

That said, best of wishes on your quest!

Comment author: Eliezer_Yudkowsky 27 April 2009 11:42:45PM 14 points [-]

Hm? Brennan isn't changing his desires, he's simply noticing them.

Comment author: Psy-Kosh 28 April 2009 01:23:08AM *  18 points [-]

I think cousin_it was going for "having one's desires feel strongly, proportional to the amount they really want it", rather than partly saying "I really want X, but my more immediate desires that I don't 'really want' are desires I feel/experience much more strongly and end up being something I want to but find difficulty struggling against."

ie, I'm guessing the point was ultimately an akrasia issue.

Comment author: thomblake 27 April 2009 11:11:42PM 36 points [-]

Eliezer,

Write more like this.

Comment author: ciphergoth 28 April 2009 05:01:31PM 13 points [-]

I feel the opposite way, I have to confess.

Comment author: JulianMorrison 28 April 2009 08:28:08AM 10 points [-]

Seconded. I can see how it won't be your highest priority, but if you have a spare moment and are looking for something to break the monotony...

Comment author: FiftyTwo 19 December 2011 09:01:43PM *  8 points [-]

Does the "If you don't know what you need, take power" quote have any origin before this? I searched for it but only found it in a post on heuristics that linked back here. I would be interested in more discussion around it before I adopt it as a method.

Comment author: Luke_A_Somers 13 May 2013 03:32:28PM 3 points [-]

Reminds me of The Left Hand of Darkness - one bit where the main character says something along the lines of, 'If action ceases to be profitable, gather information.' but there were two layers to it, with something else to do either between the two or after gathering information.

Comment author: Vaniver 13 May 2013 03:48:19PM 11 points [-]

As they say in the Ekumenical School, when action grows unprofitable, gather information; when information grows unprofitable, sleep.

Comment author: MrShaggy 27 April 2009 10:01:23PM 5 points [-]

Brennan asked: "Is this the only way in which Bayesian masters come to be, sensei?"

And thought: "How could Jeffreyssai possibly have known before Brennan knew himself?"

He wants to find a better way to train Bayesian masters.

Comment author: orthonormal 27 April 2009 11:08:43PM 12 points [-]

I'd wager 10 karma points against 1 that this is not the desire Eliezer has imagined for Brennan.

I'd only wager 5 against 1 that he has a specific desire imagined for Brennan.

(Thus begins the prediction market for the Bardic Conspiracy...)

Comment author: Eliezer_Yudkowsky 27 April 2009 11:41:58PM 11 points [-]

You win, naturally.

I can think of a desire that Brennan might have, and indeed had it in mind at the time; but since it doesn't appear in the story, it would seem that my particular belief is not particularly privileged...

...but I agree with you that Brennan's desire is almost certainly not "a better way to train Bayesian masters", that would be way the hell out of his revealed character.

Comment author: MBlume 28 April 2009 12:38:46AM 17 points [-]

I'd wager 10 karma points against 1

low priority, I'm sure, but I'd be entirely in favor of a means by which we could enact bets like these.

Comment author: gjm 28 April 2009 08:54:56AM 1 point [-]

Might be too easily gamed. Imagine that A and B are both keen to get more karma -- of course A and B might really be the same person, though not the same LW-user -- then they both make 1000-to-1 bets against one another, etc. (Of course the karma system can already be subverted a little by simpler means -- A and B just upvote one another -- but that's much slower and milder.)

Comment author: MBlume 28 April 2009 09:00:21AM 9 points [-]

I was assuming that karma was actually being transferred, zero-sum.

Comment author: pangloss 28 April 2009 09:05:48AM 1 point [-]

That puts people with a great deal of Karma in a much better position with respect to Karma gambling. You could take us normal folk all-in pretty easily.

Comment author: MBlume 28 April 2009 09:35:49AM 4 points [-]

well, I certainly wouldn't expect anyone to take a bet which could lose them their posting ability, for example.

Comment author: gjm 28 April 2009 09:10:25AM *  0 points [-]

[EDIT, later: This comment is simply wrong; I wasn't thinking straight. Sorry.]

You can't do that for uneven bets, like orthonormal's. (I suppose you could have a negative-sum system where you say "I'm willing to gain 1 point at the risk of N", and then you need to find N people who will all bet with you on those terms; they all make 1-for-1 bets, but if you win you only get 1. But that doesn't seem terribly appealing.)

Comment author: MBlume 28 April 2009 09:27:54AM 19 points [-]

I don't understand the problem.

If Eliezer imagined Brennan as wanting to create more Bayesian masters, ortho would lose 10 points, which MrShaggy would gain. Under the reverse case, ortho gains a point, MrShaggy loses one.

Comment author: gjm 28 April 2009 10:49:58AM 7 points [-]

The problem is that I'm an idiot and misunderstood; sorry.

Comment author: Eliezer_Yudkowsky 28 April 2009 05:48:36PM 15 points [-]

I voted this down and the parent up because, while it's a fine apology, you should not actually get more karma for admitting a mistake than the person who corrected you gets.

Comment author: gjm 28 April 2009 07:00:43PM 7 points [-]

Fine with me. (I'm going to take the self-flattering route and assume that my comment got voted up because being prepared to admit one's errors is a good thing, rather than because the observation "gjm is an idiot" is particularly worthy of upvotes...)

Comment author: pangloss 28 April 2009 06:07:58PM *  7 points [-]

I voted this down, and the immediate parent up, because recognizing one's errors and acknowledging them is worthy of Karma, even if the error was pointed out to you by another.

Comment author: pangloss 28 April 2009 06:16:01PM 1 point [-]

I guess this raises a different question: I've been attempting to use my up and down votes as a straight expression of how I regard the post or comment. While I can't guarantee that I am never drawn to inadvertently engage in corrective voting (where I attempt to bring a post or comment's karma in line with where I think it should be in an absolute sense or relative to another post), it seems as though this is your conscious approach.

What are the advantages/disadvantages or the two approaches?

Comment author: MrShaggy 27 April 2009 11:24:36PM 0 points [-]

Also note: "Power, he'd sought at first. Strength to prevent a repetition of the past. "If you don't know what you need, take power" - so went the proverb. He had gone first to the Competitive Conspiracy, then to the beisutsukai."

But the teacher has promised failure as a seemingly necessary step to mastery on this path, so it has not fulfilled what he went there for yet.

Comment author: sakranut 04 December 2012 04:48:07AM 2 points [-]

Shir L'Or

Nice use of Hebrew.

Comment author: alicey 16 March 2015 06:53:04PM 0 points [-]

i am curious what the nice use of hebrew is!

Comment author: Mass_Driver 17 March 2015 05:12:54AM 1 point [-]

It's probably "Song of Light," or if you want a more literal translation, "Hymn to Light."

Comment author: hairyfigment 16 March 2015 07:59:00PM 0 points [-]

I can only find 'Singing Towards Awake'. But for all I know, that's grammatical.

Comment author: Akiryx 12 September 2016 05:00:56AM 0 points [-]

Is this a continuation of something or is this just a one-off?

Comment author: Good_Burning_Plastic 12 September 2016 10:22:09AM 1 point [-]

The previous stories are here.

Comment author: AshwinV 18 March 2015 08:38:01AM 0 points [-]

This post just replaced the third alternative as my all-time favourite.