All of Dynamically_Linked's Comments + Replies

Eliezer, after you realized that attempting to build a Friendly AI is harder and more dangerous than you thought, how far did you back-track in your decision tree? Specifically, did it cause you to re-evaluate general Singularity strategies to see if AI is still the best route? You wrote the following on Dec 9 2002, but it's hard to tell whether it's before or after your "late 2002" realization.

I for one would like to see research organizations pursuing human intelligence enhancement, and would be happy to offer all the ideas I thought up for hum... (read more)

My view is similar to Robin Brandt's, but I would say that technological progress has caused the appearance of moral progress, because we responded to past technological progress by changing our moral perceptions in roughly the same direction. But different kinds of future technological progress may cause further changes in orthogonal or even opposite directions. It's easy to imagine for example that slavery may make a comeback if a perfect mind control technology was invented.

Aaron, statistical mechanics also depend on particle physics being time-reversible, meaning that two different microstates at time t will never evolve to the same microstate at time t+1. If this assumption is violated then entropy can decrease over time.

Is there some reason why time-reversibility has to be true?

If we can imagine a universe where entropy can be made to decrease, then living beings in it will certainly evolve to take advantage of this. Why shouldn't it be the case that human beings are especially good at this, and that is what they are being used for by the machines?

Constant, if moral truths were mathematical truths, then ethics would be a branch of mathematics. There would be axiomatic formalizations of morality that do not fall apart when we try to explore their logical consequences. There would be mathematicians proving theorems about morality. We don't see any of this.

Isn't it simpler to suppose that morality was a hypothesis people used to explain their moral perceptions (such as "murder seems wrong") before we knew the real explanations, but now we find it hard to give up the word due to a kind of memetic inertia?

For those impatient to know where Eliezer is going with this series, it looks like he gaves us a sneak preview a little more than a year ago. The answer is morality-as-computation.

Eliezer, hope I didn't upset your plans by giving out the ending too early. When you do get to morality-as-computation, can you please explain what exactly is being computed by morality? You already told us what the outputs look like: "Killing is wrong" and "Flowers are beautiful", but what are the inputs?

Constant wrote: So one place where one could critique your argument is in the bit that goes: "conditioned on X being the case, then our beliefs are independent of Y". The critique is that X may in fact be a consequence of Y, in which case X is itself not independent of Y.

Good point, my argument did leave that possibility open. But, it seems pretty obvious, at least to me, that game theory, evolutionary psychology, and memetics are not contingent on anything except mathematics and the environment that we happened to evolve in.

So if I were to draw ... (read more)

And to answer Obert's objection that Subhan's position doesn't quite add up to normality: before we knew game theory, evolutionary psychology, and memetics, nothing screened off our moral perceptions/intuitions from a hypothesized objective moral reality, so that was perhaps the best explanation available, given what we knew back then. And since that was most of human history, it's no surprise that morality-as-given feels like normality. But given what we know today, does it still make sense to insist that our meta-theory of morality add up to that normality?

Subhan: "You're not escaping that easily! How does a universe in which murder is wrong, differ from a universe in which murder is right? How can you detect the difference experimentally? If the answer to that is 'No', then how does any human being come to know that murder is wrong?" ... Obert: "Because it seems blue, just as murder seems wrong. Just don't ask me what the sky is, or how I can see it."

But we already know why murder seems wrong to us. It's completely explained by a combination of game theory, evolutionary psychology,... (read more)

Why is it a mystery (on the morality-as-preferences position) that our terminal values can change, and specifically can be influenced by arguments? Since our genes didn't design us with terminal values that coincide with its own (i.e., "maximize inclusive fitness"), there is no reason why they would have made those terminal values unchangeable.

We (in our environment of evolutionary adaptation) satisfied our genes' terminal value as a side-effect of trying to satisfy our own terminal values. The fact that our terminal values respond to moral argum... (read more)

Why doesn't Zaire just divide himself in half, let each half get 1/4 of the pie, then merge back together and be in possession of half of the pie?

Or, Zaire might say: Hey guys, my wife just called and told me that she made a blueberry pie this morning and put it in this forest for me to find. There's a label on the bottom of the plate if you don't believe me. Do you still think 'fair' = 'equal division'?

Or maybe Zaire came with his dog, and claims that the dog deserves an equal share.

I appreciate the distinction Eliezer is trying to draw between the object level and the meta level. But why the assumption that the object-level procedure will be simple?

Notice how nobody is willing to admit under their real name that they might do something traditionally considered "immoral". My point is, we can't trust the answers people give, because they want to believe, or want others to believe, that they are naturally good, that they don't need moral philosophies to tell them not to cheat, steal, or murder.

BTW, Eliezer, I got the "enemies list" you sent last night. Rest assured, my robot army will target them with the highest priority. Now stop worrying, and finish that damn proof already!

Seriously, most moral philosophies are against cheating, stealing, murdering, etc. I think it's safe to guess that there would be more cheating, stealing, and murdering in the world if everyone became absolutely convinced that none of these moral philosophies are valid. But of course nobody wants to publicly admit that they'd personally do more cheating, stealing, and murdering. So everyone is just responding with variants of "Of course I wouldn't do anything different. No sir, not me!"

Except apparently Shane Legg, who doesn't seem to mind the world knowing that he's just waiting for any excuse to start cheating, stealing, and murdering. :)

Eliezer, I've got a whole set of plans ready to roll, just waiting on your word that the final Proof is ready. It's going to be bloody wicked... and just plain bloody, hehe.

Nick, here's what Judea Pearl wrote on this topic. On page 59 of his book:

This suggests that the consistent agreement between physical and statistical times [i.e., the direction of time and the direction of causality] is a byproduct of the human choice of linguistic primitives and not a feature of physical reality. ... Pearl and Verma (1991) speculated that this preference represents survival pressure to facilitate prediction of future events, and that evolution has evidently ranked this facility more urgent than that of finding hindsighted explanation for... (read more)

RI, what if I wanted to buy two windows such that one is twice the mass of the other. Is that still cheating?

Nick, how would you transform my causal hypothesis (in the comment above) with intramoment dependencies into one without?

This definition of causality doesn't seem to work, since the universe clearly doesn't generate future values independently of each other. Consider the following story:

On Monday I decide to buy 2 windows of the same mass. Suppose I want to buy the biggest windows I can afford, and I have money in two bank accounts that I can use for this purpose. On Tuesday a couple of cute little vandals break both of my windows. Some of the glass falls inside my home, and rest outside. Now let:

L1 = how much money I had in bank 1 L2 = how much money I had in bank 2 M1 = ma... (read more)

I went back to the beginning of this series of posts, and found this introduction:

I think I must now temporarily digress from the sequence on zombies (which was a digression from the discussion of reductionism, which was a digression from the Mind Projection Fallacy) in order to discuss quantum mechanics. The reasons why this belongs in the middle of a discussion on zombies in the middle of a discussion of reductionism in the middle of a discussion of the Mind Projection Fallacy, will become apparent eventually.

Eliezer, would you mind telling us the reaso... (read more)

Barbour is proposing something quite different from the block universe. I'm not sure if Eliezer is missing the point, or just not carrying it across. Barbour is speculating that if we solve the Wheeler-DeWitt equation, we'll get a single probability distribution over the configuration space of the universe, and all of our experiences can be explained using this distribution alone. Specifically, we don't need a probability distribution for each instant of time, like in standard QM.

I think Eliezer's picture with the happy faces is rather misleading, if it's ... (read more)

This abstract of one of Barbour's papers may be helpful for those wondering (like me) how exactly Barbour was proposing to get rid of "t":

http://www.iop.org/EJ/abstract/0264-9381/11/12/006

Abstract. A strategy for quantization of general relativity is considered in the context of the timelessness' of classical general relativity discussed in the preceding companion paper. The Wheeler--DeWitt equation (WDE) of canonical quantum gravity is interpreted as being like a time-independent Schrödinger equation for one fixed energy, the solution of which s... (read more)

I don't have a copy of Barbour's book. Maybe someone who does can check what it says about parity violation? (Never mind, I just did an Amazon search inside the book, and it contains no mention of "parity" or "chirality".)

Anyway, my understanding is that parity violation means that reversing left and right of the entire universe would not give you the same internal experience. If this is hard to imagine, suppose that the laws of physics were such that right-handed DNA works the same as in our universe, but left-handed DNA is 10% less st... (read more)

But if you could learn to visualize the relative configuration space, then, so long as you thought in terms of those elements of reality, it would no longer be imaginable that Mach's Principle could be false.

If one learned to think only in terms the relative configuration space, it would also become impossible to imagine that parity violation could be possible, since the left-hand and right-hand versions of a system have the same relative distances. Yet the weak nuclear force does violate parity.

Eliezer, I think your (and Robin's) intuition is off here. Configuration space is so vast, it should be pretty easy for a small blob of amplitude to find a hiding place that is safe from random stray flows from larger blobs of amplitude.

Consider a small blob in my proposed experiment where the number of 0s and 1s are roughly equal. Writing the outcomes on blackboards does not reduce the integrated squared modulus of this blob, but does move it further into "virgin territory", away from any other existing blobs. In order for it to be mangled by st... (read more)

Robin, can you offer some intuitive explanation as to why defense against world mangling would be difficult? From what I understand, a larger blob of amplitude (world) can mangle a smaller blob of amplitude only if they are close together in configuration space. Is that incorrect? If those "secure storage facilities" simply write the quantum coin toss outcomes in big letters on some blackboards, which worlds will be close enough to be able to mangle the worlds that violate Born's rule?

Robin Hanson suggests that if exponentially tinier-than-average decoherent blobs of amplitude ("worlds") are interfered with by exponentially tiny leakages from larger blobs, we will get the Born probabilities back out.

Shouldn't it be possible for a tinier-than-average decoherent blobs of amplitude to deliberately become less vulnerable to interference from leakages from larger blobs, by evolving itself to an isolated location in configuration space (i.e., a point in configuration space with no larger blobs nearby)? For example, it seems that we ... (read more)

Has anyone read Learning Bayesian Networks by Richard E. Neapolitan? How does it compare with Judea Pearl's two books as an introduction to Bayesian Networks? I'm reading Pearl's first book now, but I wonder if Neapolitan's would be better since it is newer and is written specifically as a textbook.

Eliezer, the US killed at least a million Japanese in World War 2, while the attack at Pearl Harbor killed less than 2500. Maybe it is true that the US response to 9/11 is "greater than the appropriate level, whatever the appropriate level may be" but I don't think you have showed that to actually be the case.

So, what about the notion of mathematical proof? Anyone want to give a shot at explaining how that can be regenerated?

The issue is replication with variation and the necessary historical consequences of this.

Evolution requires more than replication with variation. It needs differential replication with variation.

There is therefore no way to avoid the consequences of evolution: they are not biological consequences, but consequences of the laws of physics and logic. There is no way around them.

I can think of a couple of potential ways to avoid the consequences of evolution, by attacking the "differential" part.

1. The Singleton.

2. Some other method for achieving abso