rwallace

Why safety is not safe

June 14, 3009 Twilight still hung in the sky, yet the Pole Star was visible above the trees, for it was a perfect cloudless evening. "We can stop here for a few minutes," remarked the librarian as he fumbled to light the lamp. "There's a stream just ahead." The driver grunted assent as he pulled the cart to a halt and unhitched the thirsty horse to drink its fill. It was said that in the Age of Legends, there had been horseless carriages that drank the black blood of the earth, long since drained dry. But then, it was said that in the Age of Legends, men had flown to the moon on a pillar of fire. Who took such stories seriously? The librarian did. In his visit to the University archive, he had studied the crumbling pages of a rare book in Old English, itself a copy a mere few centuries old, of a text from the Age of Legends itself; a book that laid out a generation's hopes and dreams, of building cities in the sky, of setting sail for the very stars. Something had gone wrong - but what? That civilization's capabilities had been so far beyond those of his own people. Its destruction should have taken a global apocalypse of the kind that would leave unmistakable record both historical and archaeological, and yet there was no trace. Nobody had anything better than mutually contradictory guesses as to what had happened. The librarian intended to discover the truth. Forty years later he died in bed, his question still unanswered. The earth continued to circle its parent star, whose increasing energy output could no longer be compensated by falling atmospheric carbon dioxide concentration. Glaciers advanced, then retreated for the last time; as life struggled to adapt to changing conditions, the ecosystems of yesteryear were replaced by others new and strange - and impoverished. All the while, the environment drifted further from that which had given rise to Homo sapiens, and in due course one more species joined the billions-long roll of the dead. For what was by

60Jun 14, 2009

rwallace

Message

1814

612

17y

Design, Implement and Verify

Taking a look at the latest here after a hiatus, I notice there is once again a lot of discussion about the problem of AI safety, clearly a cause for concern to people who believe it to be an existential threat. I personally think AI safety is not an existential...

Apr 13, 202232

Polarized gamma rays and manifest infinity

Most people (not all, but most) are reasonably comfortable with infinity as an ultimate (lack of) limit. For example, cosmological theories that suggest the universe is infinitely large and/or infinitely old, are not strongly disbelieved a priori. By contrast, most people are fairly uncomfortable with manifest infinity, actual infinite quantities...

Jul 30, 201127

Rational entertainment industry?

By "the industry" in this post, I refer to that part of the entertainment industry which: 1. Produces movies, TV and video games (as opposed to books, comics etc.) 2. Is motivated by profit (as opposed to fun, politics etc.) 3. Consists of companies (as opposed to lone developers, student...

Dec 10, 20105

Compatibilism in action

A practical albeit fictional application of the philosophical conclusion that free will is compatible with determinism came up today in a discussion about a setting element from the role-playing game Exalted (5:31:44 PM) Nekira Sudacne: So during the pirmodial war, one Yozi got his fetch killed and he reincarnated as...

Nov 23, 2010-5

The Curve of Capability

or: Why our universe has already had its one and only foom In the late 1980s, I added half a megabyte of RAM to my Amiga 500. A few months ago, I added 2048 megabytes of RAM to my Dell PC. The later upgrade was four thousand times larger, yet...

Nov 4, 201020

The I-Less Eye

or: How I Learned to Stop Worrying and Love the Anthropic Trilemma Imagine you live in a future society where the law allows up to a hundred instances of a person to exist at any one time, but insists that your property belongs to the original you, not to the...

Mar 28, 201044

Two probabilities

Consider the following statements: 1. The result of this coin flip is heads. 2. There is life on Mars. 3. The millionth digit of pi is odd. What is the probability of each statement? A frequentist might say, "P1 = 0.5. P2 is either epsilon or 1-epsilon, we don't know...

Feb 15, 20101

Load More (7/10)

LESSWRONG
LW

LESSWRONG
LW

rwallace

rwallace

rwallace

Why safety is not safe

The I-Less Eye

Design, Implement and Verify

Polarized gamma rays and manifest infinity

rwallace

Design, Implement and Verify

Polarized gamma rays and manifest infinity

Rational entertainment industry?

Compatibilism in action

The Curve of Capability

The I-Less Eye

Two probabilities

Why safety is not safe

The I-Less Eye

Design, Implement and Verify

Polarized gamma rays and manifest infinity

Design, Implement and Verify

Polarized gamma rays and manifest infinity

Rational entertainment industry?

Compatibilism in action

The Curve of Capability

The I-Less Eye

Two probabilities