Abstracted Idealized Dynamics

17 Eliezer_Yudkowsky 12 August 2008 01:00AM

Followup toMorality as Fixed Computation

I keep trying to describe morality as a "computation", but people don't stand up and say "Aha!"

Pondering the surprising inferential distances that seem to be at work here, it occurs to me that when I say "computation", some of my listeners may not hear the Word of Power that I thought I was emitting; but, rather, may think of some complicated boring unimportant thing like Microsoft Word.

Maybe I should have said that morality is an abstracted idealized dynamic.  This might not have meant anything to start with, but at least it wouldn't sound like I was describing Microsoft Word.

How, oh how, am I to describe the awesome import of this concept, "computation"?

Perhaps I can display the inner nature of computation, in its most general form, by showing how that inner nature manifests in something that seems very unlike Microsoft Word—namely, morality.

Consider certain features we might wish to ascribe to that-which-we-call "morality", or "should" or "right" or "good":

• It seems that we sometimes think about morality in our armchairs, without further peeking at the state of the outside world, and arrive at some previously unknown conclusion.

Someone sees a slave being whipped, and it doesn't occur to them right away that slavery is wrong.  But they go home and think about it, and imagine themselves in the slave's place, and finally think, "No."

Can you think of anywhere else that something like this happens?

continue reading »

The Meaning of Right

30 Eliezer_Yudkowsky 29 July 2008 01:28AM

Continuation of:  Changing Your Metaethics, Setting Up Metaethics
Followup toDoes Your Morality Care What You Think?, The Moral Void, Probability is Subjectively Objective, Could Anything Be Right?, The Gift We Give To Tomorrow, Rebelling Within Nature, Where Recursive Justification Hits Bottom, ...

(The culmination of a long series of Overcoming Bias posts; if you start here, I accept no responsibility for any resulting confusion, misunderstanding, or unnecessary angst.)

What is morality?  What does the word "should", mean?  The many pieces are in place:  This question I shall now dissolve.

The key—as it has always been, in my experience so far—is to understand how a certain cognitive algorithm feels from inside.  Standard procedure for righting a wrong question:  If you don't know what right-ness is, then take a step beneath and ask how your brain labels things "right".

It is not the same question—it has no moral aspects to it, being strictly a matter of fact and cognitive science.  But it is an illuminating question.  Once we know how our brain labels things "right", perhaps we shall find it easier, afterward, to ask what is really and truly right.

But with that said—the easiest way to begin investigating that question, will be to jump back up to the level of morality and ask what seems right.  And if that seems like too much recursion, get used to it—the other 90% of the work lies in handling recursion properly.

(Should you find your grasp on meaningfulness wavering, at any time following, check Changing Your Metaethics for the appropriate prophylactic.)

continue reading »

The Ultimate Source

35 Eliezer_Yudkowsky 15 June 2008 09:01AM

This post is part of the Solution to "Free Will".
Followup toTimeless Control, Possibility and Could-ness

Faced with a burning orphanage, you ponder your next action for long agonizing moments, uncertain of what you will do.  Finally, the thought of a burning child overcomes your fear of fire, and you run into the building and haul out a toddler.

There's a strain of philosophy which says that this scenario is not sufficient for what they call "free will".  It's not enough for your thoughts, your agonizing, your fear and your empathy, to finally give rise to a judgment.  It's not enough to be the source of your decisions.

No, you have to be the ultimate source of your decisions.  If anything else in your past, such as the initial condition of your brain, fully determined your decision, then clearly you did not.

But we already drew this diagram:

Fwmarkov_3

continue reading »

Possibility and Could-ness

34 Eliezer_Yudkowsky 14 June 2008 04:38AM

This post is part of the Solution to "Free Will".
Followup toDissolving the Question, Causality and Moral Responsibility

Planning out upcoming posts, it seems to me that I do, in fact, need to talk about the word could, as in, "But I could have decided not to rescue that toddler from the burning orphanage."

Otherwise, I will set out to talk about Friendly AI, one of these days, and someone will say:  "But it's a machine; it can't make choices, because it couldn't have done anything other than what it did."

So let's talk about this word, "could".  Can you play Rationalist's Taboo against it?  Can you talk about "could" without using synonyms like "can" and "possible"?

Let's talk about this notion of "possibility".  I can tell, to some degree, whether a world is actual or not actual; what does it mean for a world to be "possible"?

I know what it means for there to be "three" apples on a table.  I can verify that experimentally, I know what state of the world corresponds it.  What does it mean to say that there "could" have been four apples, or "could not" have been four apples?  Can you tell me what state of the world corresponds to that, and how to verify it?  Can you do it without saying "could" or "possible"?

I know what it means for you to rescue a toddler from the orphanage.  What does it mean for you to could-have-not done it?  Can you describe the corresponding state of the world without "could", "possible", "choose", "free", "will", "decide", "can", "able", or "alternative"?

One last chance to take a stab at it, if you want to work out the answer for yourself...

continue reading »

Timeless Control

19 Eliezer_Yudkowsky 07 June 2008 05:16AM

Followup toTimeless Physics, Timeless Causality, Thou Art Physics

People hear about many-worlds, which is deterministic, or about timeless physics, and ask:

If the future is determined by physics, how can anyone control it?

In Thou Art Physics, I pointed out that since you are within physics, anything you control is necessarily controlled by physics.  Today we will talk about a different aspect of the confusion, the words "determined" and "control".

The "Block Universe" is the classical term for the universe considered from outside Time.  Even without timeless physics, Special Relativity outlaws any global space of simultaneity, which is widely believed to suggest the Block Universe—spacetime as one vast 4D block.

When you take a perspective outside time, you have to be careful not to let your old, timeful intuitions run wild in the absence of their subject matter.

In the Block Universe, the future is not determined before you make your choice.  "Before" is a timeful word.  Once you descend so far as to start talking about time, then, of course, the future comes "after" the past, not "before" it.

continue reading »

Thou Art Physics

41 Eliezer_Yudkowsky 06 June 2008 06:37AM

Followup toDissolving the Question, Hand vs. Fingers, Timeless Causality, Living in Many-Worlds

Three months ago—jeebers, has it really been that long?—I posed the following homework assignment:  Do a stack trace of the human cognitive algorithms that produce debates about 'free will'.  Note that this task is strongly distinguished from arguing that free will does, or does not exist.

Now, as expected, the notion of "timeless physics" is causing people to ask, "If the future is determined, how can our choices control it?"  The wise reader can guess that it all adds up to normality; but this leaves the question of how.

People hear:  "The universe runs like clockwork; physics is deterministic; the future is fixed."  And their minds form an causal network that looks like this:

Fwmevsphysics

Here we see the causes "Me" and "Physics", competing to determine the state of the "Future" effect.  If the "Future" is fully determined by "Physics", then obviously there is no room for it to be affected by "Me".

continue reading »

Zombies: The Movie

72 Eliezer_Yudkowsky 20 April 2008 05:53AM

FADE IN around a serious-looking group of uniformed military officers.  At the head of the table, a senior, heavy-set man, GENERAL FRED, speaks.

GENERAL FRED:  The reports are confirmed.  New York has been overrun... by zombies.

COLONEL TODD:  Again?  But we just had a zombie invasion 28 days ago!

GENERAL FRED:  These zombies... are different.  They're... philosophical zombies.

CAPTAIN MUDD:  Are they filled with rage, causing them to bite people?

COLONEL TODD:  Do they lose all capacity for reason?

GENERAL FRED:  No.  They behave... exactly like we do... except that they're not conscious.

(Silence grips the table.)

COLONEL TODD:  Dear God.

continue reading »

Dissolving the Question

44 Eliezer_Yudkowsky 08 March 2008 03:17AM

Followup toHow an Algorithm Feels From the Inside, Feel the Meaning, Replace the Symbol with the Substance

"If a tree falls in the forest, but no one hears it, does it make a sound?"

I didn't answer that question.  I didn't pick a position, "Yes!" or "No!", and defend it.  Instead I went off and deconstructed the human algorithm for processing words, even going so far as to sketch an illustration of a neural network.  At the end, I hope, there was no question left—not even the feeling of a question.

Many philosophers—particularly amateur philosophers, and ancient philosophers—share a dangerous instinct:  If you give them a question, they try to answer it.

Like, say, "Do we have free will?"

The dangerous instinct of philosophy is to marshal the arguments in favor, and marshal the arguments against, and weigh them up, and publish them in a prestigious journal of philosophy, and so finally conclude:  "Yes, we must have free will," or "No, we cannot possibly have free will."

Some philosophers are wise enough to recall the warning that most philosophical disputes are really disputes over the meaning of a word, or confusions generated by using different meanings for the same word in different places.  So they try to define very precisely what they mean by "free will", and then ask again, "Do we have free will?  Yes or no?"

A philosopher wiser yet, may suspect that the confusion about "free will" shows the notion itself is flawed.  So they pursue the Traditional Rationalist course:  They argue that "free will" is inherently self-contradictory, or meaningless because it has no testable consequences.  And then they publish these devastating observations in a prestigious philosophy journal.

But proving that you are confused may not make you feel any less confused.  Proving that a question is meaningless may not help you any more than answering it.

continue reading »

View more: Prev