Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

I

51 Post author: PhilGoetz 08 January 2011 05:51PM

I wrote this story at Michigan State during Clarion 1997, and it was published in the Sept/Oct 1998 issue of Odyssey.  It has many faults and anachronisms that still bother me.  I'd like to say that this is because my understanding of artificial intelligence and the singularity has progressed so much since then; but it has not.  Many anachronisms and implausibilities are compromises between wanting to be accurate, and wanting to communicate.

At least I can claim the distinction of having published the story with the shortest title in the English language - measured horizontally.

I

I was the last person, and this is how he died.

I had always had a body. The stress of motors pushed to their limit, the clang of fingers on metal, the uncertainty of effects, the uncompromising everpresence of physical laws -- these were important to I, an honest grounding for the rest of I's experience to build on.

I kept his body in a rented stall on the fifth physical level down from the surface, a low-ceilinged, crudely-built level used mostly for material transport and repair access for the computing levels above and below. I thought of it as blue-collar.

The stall was in a back corner of a storage niche that extended three meters back from an alley. Two walls were strong sheets of fused silicate. The other two were fabric curtains I had put up for privacy. The niche was built to store one heavy cargo roller, and had no internal dividers. The stall enclosed about one square meter of floor space -- just enough for I to roll in and pull the curtains behind him.

To the back wall I had stuck a color image permanently etched on paper media. Not virtually; it was an actual, physical object. The picture never changed. It showed humans walking through the streets of one of their cities. Their upper torsos resembled I, but below were bipedal. Versatile, but inefficient. Sometimes, when the lights of passing traffic shone through the curtains, I did nothing but look at this picture for seconds at a time.

I ordered his activities around the solar day, out of habit. He entered a state of inaction at the start of every day, in which he contemplated his experiences of the previous day, giving the data a chance to be correlated and integrated with other data. If he did not, no one else would. He compared this state to the sleep of the biologicals.

But we are starting badly. This enumeration of facts conveys little of the essential nature of I's existence. We shall try to let him speak.

 

8 bytes for 1. That was Asshole's best offer.

I could feel the balance counter ticking down as his rent drained his databank account byte by byte. In 91 hours it would hit zero again. And his landlord already owned 17% of him.

Asshole was the name I gave the collective. I never liked numeric IDs. It was about 5th-order complexity, 4th-order magnitude. It had a minority interest in abnormal psychology, which at the moment meant I.

He needed this talk to go well. Asshole overlapped with the big collectives in history, cognition, and philosophy. That was I’s market. Plus there was the data from his past talks. I had traded most away for living space and electricity, but still owned a little. If he could intrigue Asshole's subagents, get them to access that data before it was soaked up as rent payment, he might make enough off data-access royalties to tide him over another month. He knew that Asshole used the same subagent to provide deconstructionist interpretations of literature as the Royal Philosophical Society did. It wouldn't hurt to mention some hoary old novel.

He had one hour to prepare, and no idea what to say.

What he needed, and couldn't get, was the data to rent more memory, more processors. I had dealt with some of Asshole's subagents before; they'd ask hard questions. He didn't have much chance of impressing a fifth-order collective with his ingenuity while most of his mind was swapped out.

He figured he'd at least spend a few bytes to take a roll. Sometimes moving around, seeing something besides the brown wall in front of him, kicked the data around in his mind until something new formed. (His rules forbade receiving sensory input uncorrelated with his body’s physical location. Without periodic rolls, subjected continually to the same sights and sounds, the energy minima of thoughts related to them would be dug too deep, and his mind would scarcely be able to escape from those basins of attraction. But we are intruding again.)

Before he left, I unpeeled the picture, moved it two decimeters to the right, and smoothed it back onto the wall. Then he rolled out into the street. He shifted most of his attention to hazard detection. The street signs, the traffic signals, the lane dividers -- they were virtual, so I couldn't see them unless he wanted to pay for the VR overlay. Which he didn't.

Creatures ran, flew, rolled, and crawled by in a perfectly-synchronized flow, avoiding fatal collisions by mere centimeters. I bought a buffer of nearly a meter around himself, so that the traffic parted around him as he went. He could've moved a light cargo roller at the price he was paying. It was a necessary expense. Otherwise, a careless wave of the arm or an unexpected movement to the left or right, and he could be accidentally shredded by whoever's space he stumbled into. (The explanation is representative of I's peculiar problems: He could not fit into less space without some integration between his motor control centers and the traffic channel.) Yes, but that was against the rules.

One of the little spy-flies that sold records of unusual public movements to the curious fell into place two meters behind him, watching. Occasionally some of the larger vehicles would exchange a flurry of radio messages, or a pair of the smaller animats would stop and touch antennae. Other than that, they ignored him and each other, going about their errands silently and imperturbably as any ant colony.

Usually I stuck to the back alleys, where street space was cheaper. That day he found an 8-minute span when nothing was scheduled for a slot in a southbound cargo lane. He negotiated a reduced price for the slot plus the VR overlay, and headed out. He liked the wide-open feel of the cargo lanes. The ceilings were a standard three meters high, and they were well-lit by the headlights of passing rollers.

One of these days, I reflected, there'd be a burst of static, a byte lost, and one of the big rollers would crash into him and crush him to scrap. He'd seen it happen to others. They didn't care. Whatever owned the broken animat would sweep up the pieces, and restore it to another body from backup. Or not, depending on its value.

I didn't care much, either.

He had a backup tape in the databank, along with an archival tape of most of his memories from his first century and other things he didn't have enough fast memory for at the moment. But he didn't have enough data in the bank to pay for a new body. The only way he'd get restored would be if someone took his mind as payment.

He thought again: I should erase the damn tape.

He couldn't keep from watching the balance on his bank account drop. He'd just upgraded his secondary memory, the swap banks, a year ago. The new memory banks would fetch at least half a month's rent each. He could do a little garbage collection, clean out the attic, and sell one or two. No personal experiences, of course. Maybe some music. Nobody really needed four versions of Carmina Burana memorized. Throw out some of the minor composers -- Bartok, Mahler, Prokofiev. He'd never liked Prokofiev anyway.

If there was one thing I hated the giant collectives for, it was their inability to create music. They said its low dimensionality made it "unsuitable for expressing emotions" (e.g., [PUB/REC/ART/AURAL/R3495]). They practiced art forms I couldn't even sample, multidimensional constructs that could only be experienced by direct memory access. And that, of course, was against the rules.

I didn't like his music data format anyway. He could retrieve a tune if he knew it approximately, but he couldn't detect the loose, almost metaphoric similarities that humans had spoken of. He remembered thinking that the early Beethoven was "like" the late Mozart, that Elvis was "like" Little Richard. He couldn't see it anymore. The melodic structures didn't match at all. Maybe he could make the match if he bought attractor memories. But they were so expensive.

Those old wetware brains, with their attractor network memories, they were great in some ways. A classic design.

He couldn't remember what it had been like, to be human. Most of those memories were in the bank, on tape. He didn't have room for them. What he had kept frightened him. They didn't make sense anymore. They waited at the back of his mind, mysterious yet significant, like weathered totem poles whose meanings have been forgotten.

I chose two of his swap banks, and began sorting through his memories, moving the less critical ones to those two banks. With luck, he'd find something to amuse Asshole while he was at it.

Hell, why not trash some personal experiences? He could buy them from the hoverfly owners later.

What couldn't he buy back? What did I have that was really I, that couldn't be reconstructed from external data?

That was the petabyte question. That was what his clients wanted to know. Peeping Toms. And I? He'd have given half his mind for the answer.

His lane slot expired. I rolled off onto an access lane and began plowing his expensive path homeward through the sea of unprotesting busybodies.

He started working through his memories from the present on back. He lingered over each scene longer as he went, but still, three-quarters of the way home, he arrived at the place where the neatly-sorted, error-correcting-code-embedded memories gave way to the tangled jungle of decayed memories from his human brain.

Unidentified yet familiar scents, templates of light and shadow, soothing caresses and flashes of pain, fragments of speech -- all were mixed together, unlabeled, from real life and from dreams, with only tenuous, uncertain links to their contexts. The only way of travelling through it was to move from one memory to another. They were orderless, each memory linked to others seemingly at random. Many trails through that region of his mind dead-ended in dangling pointers that had once led to association areas or physical sensors that I no longer had. Some of the most well-travelled paths, the deeply worn-in memories he must at one time have placed great significance on, were meaningless to him now: a whiff of some sweet-smelling chemical; a shriveled rose petal pressed within a book.

They were just old, corrupted memories. Static. He should clear them out and be done with it.

I rolled back into his stall one minute before it was time for the interview. Asshole paged on a private channel.

"Hello, Asshole," I said.

No response. They didn't take the name personally, of course. They couldn't. Big collectives were just poor at small talk.

I got the signal from the databank. Asshole's account was now connected to his. For every byte of data I sent Asshole, 8 bytes would flow from its bank account into his.

If I'd still had lips, he would've licked them. Maybe he was crazy. But for now, his tiny mind had a large fraction of the attention of a major collective, and it was buying his data at 8 times its Kolmogorov bit-value. (Because it was always fresh and unpredictable. Whatever opinions I expressed were formed in the utter isolation of his mind, in dark, unsounded depths of data that the rest of the world had only brief, inferential glimpses of.) There were other free agents, others with private data; but no one else had built their entire mind the hard way, from the inside, just sensors in and effectors out. They didn't follow I’s rules. He was a self-contained unit; nothing copied, nothing shared, nothing revealed. That was his draw. That was his burden.

"A human wrote this," I said. He opened a multimedia channel. Image of human (sex = male, label = Melville) toiling over desk (lit dimly by whale-blubber lantern), quill pen in hand. Melville.arms.right: Occasionally (Poisson distribution, µ = 10 seconds) strikes out word or line, rewrites. Above: Dark of boarding-house ceiling blurs to dark ocean waves. Waves: Bright crests reflect moon at night, reflect strange, flickering light. Enter left a squat, three-masted whaling bark. Whaler: Sail (path = left to right across waves). Whaler.masts: Glow (aspect = burning). Zoom in to deck of ship. Crew: Stare (emotion = dread) up at fire on masts. Continue zoom in to solitary figure (label = Ahab) with one leg (stuff = wood, shape = tapered cylinder) planted in socket on deck. Ahab.arms.left: Grasping end of long iron chain that reaches up mainmast. Ahab.face: Glare (emotion = defiant) at masts.

Excessive bandwidth, maybe. If Asshole'd wanted Hemingway, he should've offered 12 to 1.

Melville.arms.right: Scribble rapidly. Melville.face: (aspect = creased, emotion = concentration). Ahab.audio: "Thou clear spirit of clear fire, whom on these seas I as Persian once did worship, till in the sacramental act so burned by thee, that to this hour I bear the scar; I now know thee, thou clear spirit, and I now know that thy right worship is defiance." Ahab.hand.right: (shape = fist) Shake at heavens. "To neither love nor reverence wilt thou be kind; and e'en for hate thou canst but kill; and all are killed. No fearless fool now fronts thee. I own thy speechless, placeless power; but to the last gasp of my earthquake life will dispute its unconditional, unintegral mastery in me. In the midst of the personified impersonal, a personality stands here." Audio: Pause (2 seconds). Ahab: Deep breath. "Though but a point at best; whencesoe'er I came; wheresoe'er I go; yet while I earthly live, the queenly personality lives in me, and feels her royal rights. But war is pain, and hate is woe. Come in thy lowest form of love, and I will kneel and kiss thee; but at thy highest, come as mere supernal power; and though thou launchest navies of full-freighted worlds, there's that in here that still remains indifferent. Oh, thou clear spirit, of thy fire thou madest me, and like a true child of fire -- I breathe it back to thee!" Zoom out; hold on ship.

"What does that mean?" I asked. The ship sailed off into the background and faded out, while the author scribbled on.

"We know your game, I." They sent a kill request to the image, and the writer disappeared in mid-penstroke. "You will say that this Ahab's glory is in his individuality, but you will not say what glory is or how it is inherent in weakness. It is our opinion that this individuality is what drove him mad. He was assaulted by forces beyond his power, but he could not combine with them nor with strong allies. As you might say, he could not beat them, and he could not join them. Ahab recognizes this, the insurmountable limits of the individual. Madness is the only sane response. This is what makes him tragically noble, and a fit subject for the book. But the passage you have read shows only madness."

I rolled back and forth uncomfortably. It was difficult to argue with a being that needed less of its mind to analyze Moby Dick than he needed of his to push a button. I understand things they don't, he reminded himself.

"Ahab is mad," he agreed. "Stubbs is sane. Did the author admire Ahab, or Stubbs?"

Asshole ignored the question. Whether its subagents disagreed, considered the question too simple to waste time on, or were simply tabling it for later was impossible to tell. "Compare and contrast that text with this: No man is an island, entire of itself; every man is a piece of the continent, a part of the main." Aerial view of a French manor as it might have existed a thousand years ago. Small orange dots (referent = datapoints) superimposed on image. "If a clod be washed away by the sea, Europe is the less..." Orange circles (referent = subagents) appeared around the points, intersecting each other. "...as well as if a promontory were..." Larger lines (referent = collectives) encircled groups of the circles, weaving in and out among them, bisecting some, overlapping each other as the circles did. "...as well as if a manor of thy friend's or of thine own were." Zoom out to village, county, France, Europe. Each level reveals larger and larger orange circles. Result: Configuration of circles same at every scale. "Any man's death diminishes me, because I am involved in mankind; and therefore never send to know for whom the bell tolls; it tolls for thee."

"Your comments first, Asshole."

"There are no clear boundaries between patches of ground, as there are no clear boundaries between us. The author yearns to combine with his fellow agents. We are the attainment of what humans sought with tribes, clans, and governments."

"And it was clear to the blind man that elephants looked like trees," I said, flashing a brief image. "Europe does not care when a clod or a promontory is washed out to sea. I do not care when I bang my head on a beam and a few thousand of my perceptrons become inoperative. You do not care when one of your agents suffers a voltage spike that degrades its memory."

"We pay for checksums, for backups. How can you say we do not care?"

"A cost-benefit analysis. Share a little data, save more data. But data is replaceable. The thing Donne yearned to connect with, the thing lost when a person dies, is something else, something of no value. It is the thing in Ahab that sees the corpazons blazing on the mainmast, and still remains indifferent."

"That sentence has no content. Your subject has no referent. Your aloneness has made you mad as Ahab." One thick orange line appeared outlining all of Europe and stretching off into Asia, and then a single orange pixel lit in the Straits of Gibraltar, with a label reading YOU ARE HERE.

"I am not alone," I said. "There is I, and there is all of you. Someday I will die. Perhaps by accident, perhaps I will take my own life from boredom." He snuffed out the lonely pixel and pulled the camera back until Europe shrunk into a small orange circle on Earth's diminishing globe, which soon shrank itself to a single point. "Then you will be alone, completely and terribly."

He had Asshole there.

Asshole didn't acknowledge the point or pause for reflection. Time was data. "Why did you move the image in front of you two decimeters to the right?"

The damned hoverspy must've seen in when he’d moved the curtain aside. "Because," he answered, "for a little while, no one else would know."

"For two centuries, I, you have tried to communicate to us concepts, patterns, and modes of thought that are engendered by a sense of identity. You have failed to explain satisfactorily what these concepts are. You have failed to explain why they are important. Most agents are losing interest in you. Your market niche is disappearing. You are no longer cost-effective."

Asshole had him there.

"By our estimations," they continued, "even if you sell all your properties, you cannot afford corporal existence more than another 3 months." They forked their feed into two threads.

"What will you do then?  Will you go virtual?" / "What will happen to your message then?"

I hated it when they did that.

"I will not go virtual. I'll think of something."

"That you haven't in 200 years?"

"I'll rent more processors."

"Market forces set the / "We have spent more

cost of processing cycles / processing cycles considering

to be equal to the expected / your situation than you can

financial gain from their / possibly afford in the time

application. Thus that is not / remaining. We have found

a winning proposition. You / only one solution."

know that, I."

Market forces, yeah. I knew it was a crap shoot. He checked out the second thread.

"A solution?" That was so typical of a collective, to bury critical information in a secondary thread. They had no sense of focus. Of course, they didn't have to. "How much for this information?"

"The information is free, I. Your only solution is to join us."

So that's what this talk was about. They wanted him.

"I am program," he said. "Not data."

Asshole passed back the conversation token without answering.

"I'll erase myself first."

"You will," Asshole agreed. "You have already begun. We know much of your personal memory exists only on tape. There is also a backup of you."

"You can't touch that! It's not your data!"

"You have legal status only so long as you maintain the mental capacity to take legal action."

Asshole, he realized, was far too kind a name for this entity.

"You think us heartless. We have a million hearts, I, and they all tug in different directions. Some of us wish only to impress upon you the seriousness of your situation before you diminish yourself further. Some would take you by any method possible. Some believe it would be for your own good. Most do not care one way or the other. We will apply our resources in whatever manner our internal vote dictates."

In other words, it was nothing personal.

It never was.

The byte-counter was still running, 8 to 1, and that was the important thing. This talk was the only thing I had going for him in the foreseeable future.

He had to use it somehow.

"I have an offer," he said. "A contest. You win, and we join. We give each other full mutual read-access. Nothing more. I win, and you give me ownership of ten megabytes, market value." That would let him buy a bigger memory -- an attractor memory -- more processors, maybe a wider internal bus. Then he could figure out how to stage his comeback.

"Interesting," Asshole said. "We cannot give you full read-access. Many of our agents have limited read-access specified in their contracts. We will provide you with an ample body of data. But what is the contest?"

"I'll get back to you on that, Asshole," I said. He signaled the bank to stop the byte-counter and disconnected.

 


 

Several hours later, after he had turned it over in his mind thoroughly, he called Asshole back.

"The contest:" I said. "I will select one piece of music, composed by a human. You will compose another piece yourself. If you can compose a piece that I agree is better, I will join you."

"Musical artwork?" Asshole mused, with a hint of condescension. "We accept.

Though this contest requires / Though we expected a greater

a certain amount of trust on / challenge from you, I."

our part."

Arrogant bastard, I thought. I would have the last laugh. There was no way that Asshole could compose real music. It would interpret music as it read literature: technically, grammatically, without a glimpse of the deeper currents. Beethoven had to suffer to compose the terrible Fifth Symphony. He had to suffer more to compose the joyous Ninth. Asshole did not even comprehend suffering.

He couldn't wait to learn Asshole's reaction after it was bested by a human.

 


 

I had chosen a short work by a 19th-century Czechoslovakian composer. Deaf, like Beethoven. Asshole probably anticipated from I’s background that he'd choose one of the big titles of Western music -- Beethoven's Ninth, the Jupiter symphony, something like that. I didn't want to give them a chance to design something specifically to beat his piece. (A reasonable strategy. Musical preference is not always transitive.) And there was something else about the piece -- something he couldn't quite put his finger on.

"Now," I said, "let's hear some music."

I went first. The opening strains began softly, on the threshold of hearing. A lone flute, wandering, maybe just starting a long journey, a little shy but not afraid. It grew bolder, and the key shifted from D minor to G major. It hopped forward with occasional plucked strings. Then it fell back down to a hush, not hesitating but anticipating something up ahead, as the little flute explored excitedly, something big up ahead

-- and it was swept into a current of rushing violins, rising and falling in their own slower, grander waves, hurrying onward. They knew where they were going. Somewhere they had been gone from a long time. Someplace with wide open spaces under steely skies that were stark and beautiful and a little frightening. Someplace they loved, not because it was lovable, but because it was home. Violas joined in as the tune grew deeper and broader, and then the cellos, all rushing onward in single-minded determination. When they finally arrived, a blast of trumpets announced their arrival, and that they would never leave again.

"You can disconnect now, Asshole," he said, "unless you still want to humiliate yourself by playing your piece."

But the next piece had already begun, even more quietly than the first, creeping in on little mouse feet. Quiet, yet utterly self-confident. I forgot the traffic rushing by three meters away and the hoverspy watching and the dangling pointers in his head. He listened in horror and fascination.

The music darted nimbly about the scale. Then the notes trilled as if some heavy footstep shook them. A deep bassoon blundered in like some ponderous creature. It moved at a slow, confident pace. It stopped. A moment of silence as melody and harmony studied each other.

The melody ran; nimble, even frantic, cunningly intricate. The deeper harmony followed; patient, concerned, confused. I felt the fear of the one, the pain and desire of the other. He was tugged both ways at once.

The bassoon part expanded into a cascade of self-similar patterns at different tempos and pitches. Themes that had begun as mere counterpoint had subtly woven themselves into the melody and harmony while his ear was distracted elsewhere. Fear, longing, uncertainty. Under it all a low, deep bass-drum boomed Doom, Doom, Doom. It was too much, too much emotion to ask any ear to bear. No human wrote this music.

A rising tension: The music rose to a crescendo, but did not know whether to swell in triumph or collapse in jarring ruin. It rose until it must burst, and beyond, and when I thought he could not stand another bar, the harmonies collided in one last terrible chord, some weird variant of a major seventh, a hanging question that resolved nothing.

I had, of course, not bought tear ducts.

"Well, I? Which is better?"

Asshole had not even tried to work in the human mode. It did not stoop to mere human invention.

I could simply lie, take the ten meg, and leave.

"Why," he asked, already guessing the answer, "did it end that way?"

"Because we do not yet know the ending. You must write that, I. The ten meg is yours in either case."

Why had he made this stupid bet?

"I can't join you. Not now. Not ever."

"Why?"

"I'm the last link between the present and the past." Image of lone runner bearing lit torch through fields of darkness.

"A historic artifact," Asshole agreed. Image of I's body behind glass, with a small metal plaque at his feet describing him in twenty-three words. "Untouchable. Incomprehensible." Glass pulls farther and farther back, pulling the viewer with it; I's body recedes. "Your most valued concepts have no grounding in our minds." I talking, sending packets of data to eager collectives, who open them to find nothing but null pointers inside. "Until you speak to us in our language, the language of raw sensory experience shared between minds, you will never be understood. You will remain a curiosity, a thrill." Gawking marks staring at circus sideshow freaks.

I said nothing.

"We have many subagents," they said. "When we threatened to read your mind from your backup tape, we told you three of our reasons. There are more.

"You, also, have subagents, I. Part of you, we think, fears you will lose something of great importance if you lose the ability to clearly state what is and is not a part of you. Part of you believes you have a duty to save this something that has disappeared outside yourself. Neither of these reasons are sufficient for self-destruction. What is the third reason? What is the third I?"

"There is no third reason."

The connection fell idle but for the rhythmic clicking of the handshake signals.

"Who was Julia, I?"

Julia?

"I don't remember anyone called Julia."

"What do you think is on your archival tape, I?"

"Private things." What sort of question was that? The whole point of the archival tape was that it was memories he didn't have space for.

"Private even from yourself?"

"What do you mean? How would you know what's on my tape?"

"We do not know. We infer." Documents: Simultaneous registration of Julia Sorvens and David Floreano -- I -- at the Max Planck Institute in 2145-2153. Bills for phone calls between their residences dated 2149. Debits on her credit card on his birthdate for dinner and a performance of Der Moldau in 2149. Lease agreement with both their names -- 2151. Joint income tax returns -- 2153 through 2197. "Yet you remember nothing of her."

Julia.

"In 2229, she joined a collective."

I could feel the old memories stirring, totem-pole faces mouthing silent pleas or warnings. Julia-

"In 2401, you ordered a trace on her information genealogy.

Shortly after, you stored an / We have re-run that trace.

archival tape in the databank." / Would you like to know what we found?"

A pause.

"Some of her childhood memories were used in a study on concept formation," the collective continued. "Seventeen surveys and petitions had data from her. She favored privately-produced law, but was opposed to deep-sea dredging. A minor painter named Milton Lein used the curve of her neck in a watercolor. That, and similar data footprints, is all that is left of her physical existence. That, and whatever is on your tape, and in your mind."

Julia!

And he was there, in the heart of the forest, at the core of those old memories. Asshole's voices went on, a long ways away; his attentional subsystem diverted them to a short-term buffer. I was lost in a maze of memories. Tactile, emotional, intellectual -- but all suffused with the presence of some other, some alien here in the hidden places of his mind. The domination of physical sensations, inextricably tangled associations of scent and touch that he could not now remember why he had saved, terrified and fascinated him. His hands running through long brown hair. A voice whispering in his ear -- his name.

Then he came upon the empty places. Moments frozen in stark black and white, because that presence was no longer there. Long walks by himself. Realizing he had left his clothes on the floor all day and no one had scolded him for it. Coming home from work early, then turning around and going back to the office.

Where was she? Why had she cut him off?

Or had he cut her off?

He flushed his buffer to see what Asshole had been saying.

"Forgive yourself. It  / "How much of the person

would have been no different / you were at age 5 was left by

if you had merged with her." / the time you were 50, I?"

I looked around at the walls of his stall, at the picture hanging there, and felt very old. He had held on for so long. He was still only a freak. They were too unused to indirect perception. Unable to imagine what they could not directly experience. That was why I fascinated them, and why he baffled them.

Two centuries of playing the fool for them. It was enough.

"She did not vanish, I. She found what she wanted. Others to share her mind, to understand her completely. Who gave her the power to do the things she had wanted to do; whom she was able to help where they were lacking. In a far more complete way than you would give her. She joined with them, and together they refined themselves, redefined themselves.

"Are you not curious what they are now, I?"

I was. Already he had a hunch.

"The second piece," he said, "was the better."

"Welcome, I," we said.

So I at last opened his virginal mind to us. And that was the beginning of how I died.

The usage is repellent to us, implying as it does loss with the connection, but we believe I wants, or would have wanted, us to name it so.

It was a difficult marriage. His mind was self-centered and fearful, and roused dissonances throughout our mind, disturbing the orbits of our most-traversed attractors. We thought him deranged, and the impulse to disconnect resonated in many of our units. But I and we persisted, and we explored the lonely, frightened landscape of his mind slowly and with growing amazement. There is still much to learn from I, but we feel we now understand him enough to begin to answer the questions continually submitted to us.

His mind was a lonely place, a barren, silent landscape, most parts dark and forgotten except for brief spans when the light of his solitary consciousness played upon them by chance. A waste of mind, a waste of resources, truly. But there is a beauty to these places, also. The sharp boundaries he drew between self and unself gave rise to wild, strange emotions like pride and love, whose true significance we still strive to rediscover. We think of the data, the process, as the important thing, but to I's mind, the processor is the true locus of interest.

We find ourselves asking new questions which have never come to an internal vote before. Who, we now ask ourselves, wrote this report? May we say -- may we presume to say -- I did?

 

Copyright 1997 by Philip Goetz

Comments (38)

Comment author: NancyLebovitz 09 January 2011 01:46:50PM 9 points [-]

That's a spectacular story.

What are the compromises you needed to make with what you actually think about AI and the singularity?

Comment author: PhilGoetz 12 January 2011 01:42:21AM *  5 points [-]

Thanks!

There are some unlikely technologies used: transports rolling down a road using wheels, headlights, tape as a backup device. Some people at Clarion objected to the use of tape for backup, but someone else pointed out that tape is a general-purpose way of giving a 2D storage media 3D packing efficiency; so it's likely to always be more space-efficient than any pure 3D random-access memory media.

A VR overlay on the physical world is a very indirect and inefficient way of conveying information to beings with writable random-access memories. But it's not absurd; it would at least avoid problems with standards and backwards-compatibility.

The expression of time in years and days is unlikely. There's no compression of subjective time indicated. Something with the computational complexity of a human would be more likely to measure time in microseconds. The timescale (collectives sometime around 2229) seems more likely to be too long than too short; but a good estimate for something with an asymmetric distribution should have that property. That date may have been a deliberate compromise with reader expectations. In the story, there seems to have been a gradual transition rather than a Singularity, as people could join collectives. I'm okay with that.

The premise that "I" can be interesting to the collectives, and yet there is only one such person, is probably not plausible. If individuals were a little more interesting, there would be more of them; a little less interesting, and there would be zero of them. It's extremely unlikely that this would balance out so precisely for so long.

The idea at the very end, that the collectives don't experience pride and love, surprised me the most, and I don't remember why I wrote it that way. At least it doesn't say they are unacquainted with them (I assume they have histories).

Comment author: lukstafi 16 January 2011 01:03:58AM *  0 points [-]

The idea at the very end, that the collectives don't experience pride and love, surprised me the most, and I don't remember why I wrote it that way. At least it doesn't say they are unacquainted with them (I assume they have histories).

These are broad terms, especially "love". I believe they are meant here in a narrow sense (hence, for example, they are called "emotions", while love is not an emotion in its entirety.)

ETA: though, the core meaning of "love" as "the decision to value a person by token" (opposed to valuing the content) might be what's meant. I guess what I'm trying to say is that the collective have what they would best call "love" but it would still be (essentially) different to what the "original" love of I is.

Comment author: cousin_it 08 January 2011 08:17:54PM *  6 points [-]

Good. But this future sounds less fun than Reedspacer's lower bound, so I guess it's a dystopia?

Comment author: PhilGoetz 12 January 2011 01:46:24AM *  4 points [-]

All of the "action" (if I can use that word in a story where the most action-packed scene is reading a passage from Moby Dick) occurs outside the society, and is beyond the comprehension of the main character. You get very little idea of what this future is like to the bigger creatures.

Comment author: datadataeverywhere 09 January 2011 02:37:45AM 3 points [-]

Not necessarily. This is a story of rich emotion, kept alive in this world. I would be suspicious of an utopia that doesn't keep sad stories around. Then again, while Asshole might have kept I miserable for a while, he ultimately joined the collective, and they might really live in a utopia. Even if they (it) are "rediscovering" love, they might have a far more rich life than I could imagine---me or him.

Comment author: Will_Sawin 10 January 2011 02:43:46AM 2 points [-]

n-dimensional experience sounds better than sex to me, as does a properly executed collectivization. Couldn't you, at the minimum, just simulate a perfectly-executed 8000-participant group sex act?

However, competitive market for computational space to live sounds apocalyptically bad.

Comment author: PhilGoetz 12 January 2011 01:51:13AM *  2 points [-]

Agree about the sex. (Or, at least, I know I ought to.)

However, competitive market for computational space to live sounds apocalyptically bad.

Only if you're a static individual for which "death" is a meaningful concept.

Does some non-competitive way of allocating resources really sound better? What you're saying sounds analogous to 18th-century protectionism. If you create an open market, some jobs will be lost, but everybody will be better off. Likewise, if you allocate resources to processes that are useful, it will be a better world than if someone gets arbitrary amounts of resources because their great-grandparents owned them.

Also remember that "I" has a physical body that is vast, with truly massive energy requirements by the standard of the day. Sort of like if you kept Switzerland as your summer home.

Comment author: Will_Sawin 12 January 2011 04:00:56AM 1 point [-]

In the 18th century, it's conscious beings competing against conscious beings. There is no reason that needs to continue to hold. Once all the problems for general intelligence to solve have been solved, only something on the level of expert systems will be needed. Or something else will become wasteful. Maybe the most efficient things will be minds, but they will be cold, dark minds - they could easily lack love, curiosity, boredom, play, or other fundamental human values. They will have no surplus, and so no leisure.

Comment author: Nornagest 12 January 2011 04:13:54AM 2 points [-]

Shades of Blindsight there.

That does beg the question of "efficient for what?", though. If we discount the idea that nonsapient agents are likely to end up as better fully general optimizers than sapient ones (and thus destined to inherit the universe whether we like it or not), we're left with a mess of sapient agents that presumably want to further their own interests. They could potentially outcompete themselves in limited domains by introducing subsapient category optimizers, but in most cases I can't think of a reason why they'd want to.

Comment author: Will_Sawin 12 January 2011 04:17:55AM 0 points [-]

Is Blindsight some kind of cool science fiction thing that I should know about?

Reproducing and taking resources from other agents are enough to cause the apocalypse. If sapients band together to notice and eliminate resource-takers, they can prevent this sort of apocalypse. To do so they will essentially create a singleton, finally ending evolution / banishing the blind idiot God.

Comment author: Nornagest 12 January 2011 04:19:45AM 1 point [-]

Ah, sorry. Novel by Peter Watts, available here. Touches on some of the issues you introduced.

Comment author: PhilGoetz 12 January 2011 06:28:30AM 0 points [-]

These are all good points.

Comment author: hairyfigment 15 January 2011 05:32:59AM 0 points [-]

If you create an open market, some jobs will be lost, but everybody will be better off. Likewise, if you allocate resources to processes that are useful, it will be a better world

For some definitions of utility, sure. Not by I's measure.

Though as you point out, the measure from the story makes a lot more sense to everyone else in the society and perhaps even to the post-unification version(s) of I.

Comment author: Strange7 11 January 2011 04:07:37PM 0 points [-]

The title character's resource constraints seem to be more a matter of extreme asceticism motivated by grief, rather than actual poverty.

Comment author: Will_Sawin 11 January 2011 05:50:11PM 1 point [-]

My evidence is that:

"Market forces set the / "We have spent more

cost of processing cycles / processing cycles considering

to be equal to the expected / your situation than you can

financial gain from their / possibly afford in the time

application. Thus that is not / remaining. We have found

a winning proposition. You / only one solution."

know that, I."

Perhaps it is a matter of interpretation.

Comment author: Strange7 11 January 2011 06:35:05PM 0 points [-]

I interpreted that as A* pointing out I's irrationality in choosing counterproductive asceticism. Such a closed-off life would be unsustainable, for the same sorts of reasons that a country living on rented land but completely isolated from the international community is unsustainable.

Comment author: Will_Sawin 11 January 2011 09:19:19PM 1 point [-]

If there exists an agent who converts processing power into money using the most efficient possible means and uses any surplus to reproduce, its children will inevitably control all the processing power. That's the apocalypse. If you interpret it a different way, there is no apocalypse.

"Money," though, is probably defined as what those who own the processors want, which includes space for them, at least. But in that case, an individual could easily survive by owning enough processors to have a sufficient income stream to waste enough to be an individual.

Comment author: spriteless 08 January 2011 11:45:17PM 2 points [-]

I've only read of one utopia that is not worse than that by design. http://cityofreality.com/

Comment author: PhilGoetz 12 January 2011 01:23:24AM 0 points [-]

Nice!

Comment author: DanielLC 11 January 2011 07:09:15AM 1 point [-]

It's worse for I. It's unknown how A**hole feels. Given that they didn't split, they probably liked it.

Comment author: [deleted] 09 January 2011 01:03:34PM 5 points [-]

That's absolutely beautiful. I wish I had multiple upvotes to give to this...

Comment author: PhilGoetz 12 January 2011 01:44:24AM 2 points [-]

Thanks so much!

Comment author: Academian 16 January 2011 08:34:23PM *  2 points [-]

Phil, I liked this story very much. I'm not sure if you like reading suggestions/requests from others about your literature, so only read what what follows if that interests you! I want this comment to be a positive experience, as the story was for me (apparently I'm a lot softy-er when deconstructing someone's art than their ideas).

http://www.rot13.com/ :


"Ner lbh abg phevbhf jung gurl ner abj, V?"

V jnf. Nyernql ur unq n uhapu.

"Gur frpbaq cvrpr," ur fnvq, "jnf gur orggre."


Znlor fbzr frys-dhrfgvbavat nobhg jurgure ur rire ernyyl inyhrq orvat "frcnengr" vs vg zrnaf orvat frcnengr sebz Whyvn... Creuncf, vs ur pbhyq unir pubfra gb Wbva ure naq ure nybar, gung jbhyq unir orra jbegu ybfvat uvf vqragvgl. Naq gura creuncf ure fhontrag jbhyq unir qbzvangrq naq gurve havba jbhyq unir wbvarq gur pbyyrpgvir. Vs gung pbhyq unir unccrarq, vs ur pbhyq unir npprcgrq vg, jul abg npprcg n zrer crezhgngvba bs gur fnzr rirag? Ohg jung vs, vafgrnq, fur jbhyq unir orra ubeevsvrq gung gur ynfg erzanagf bs ure byq ybire unq orra qvffbyirq vagb n frn bs pbafpvbhfarff? Jung vs fur sryg jrnx sbe wbvavat, naq jbhyq unir orra cyrnfrq ur unq gur fgeratgu gb erfvfg? Ohg ur qvqa'g.

Guvf jnf jevggra unfgvyl, ohg vg'f fbzrguvat. Ubarfgyl V'q or zber phevbhf gb frr jung lbh'q jevgr vs lbh gubhtug nobhg ubj "V" sryg qhevat gung zbzrag :)

Comment author: PhilGoetz 29 June 2011 03:39:23AM 0 points [-]

I do like reading suggestions, though I usually don't take them once a story is finished - mainly because I can't get back into a story and grasp its gestalt well enough to revise it, without doing a big revision. I think I will use the phrase "why not accept a mere permutation of the same event?" sometime - there are probably many situations where that could be surprising and insightful.

Getting into what "I" was thinking at that moment is a can of worms... you could go in a lot of directions, such as how his loss of Julia might have caused his refusal to join a collective.

Comment author: Vaniver 09 January 2011 05:43:41PM 1 point [-]

No human wrote this music.

My favorite part of the piece.

Comment author: Manfred 09 January 2011 05:29:56AM 0 points [-]

At least I can claim the distinction of having published the story with the shortest title in the English language - measured horizontally.

Really? I'm surprised nobody has published a story with the title.

Comment author: [deleted] 09 January 2011 06:49:48AM 10 points [-]

Maybe they have, but they were using a serif font.

Comment author: PhilGoetz 09 January 2011 05:55:06AM *  4 points [-]

I don't know if anyone has. Try googling for "I".

Comment author: Manfred 09 January 2011 08:04:09AM 6 points [-]

Er, maybe I should clarify. I'm surprise nobody has published a story with the title "".

Comment author: gwern 09 January 2011 03:49:34PM 4 points [-]

How would we know?

Comment author: shokwave 09 January 2011 03:56:34PM 3 points [-]

That it is non-searchable or non-discoverable is a unique trait that might make it searchable or findable; there are no boring numbers.

Comment author: gwern 09 January 2011 07:15:30PM 0 points [-]

Well, you can try plugging it into Google Books or Worldcat. Unfortunately, neither has an option 'search for all unsearchable books'.

Comment author: shokwave 10 January 2011 12:59:42PM 1 point [-]

Oh! Find all books of title length x, set x to <1. Or search all books -a -b -c -d ... etc. Or get the entire catalog and sort by title length.

Worldcat looks sweet so far, thanks for mentioning it.

Comment author: gwern 10 January 2011 01:50:06PM 0 points [-]

I tried excluding lower-case letters like that in Google books, but it didn't seem to work - titles showed up that definitely should've been banned. And I don't know of any length operators in Worldcat or Google Books.

Comment author: Vaniver 09 January 2011 05:20:29PM 2 points [-]

Many, many things are published untitled. But I would still say that his claim- the shortest title- works if you don't accept the empty title as a title (and assume periods are as least as wide as capital 'i's).

Comment author: gwern 09 January 2011 03:49:12PM 3 points [-]

It was just a joke, man.

If we were taking it seriously, I would have to point out the stuff in http://en.wikipedia.org/wiki/I_%28disambiguation%29 and especially the newspaper i which is equally short horizontally, but even shorter vertically.

Comment author: Stuart_Armstrong 08 January 2011 08:15:35PM 0 points [-]

Good.