Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Causal Reference

28 Post author: Eliezer_Yudkowsky 20 October 2012 10:12PM

Followup to:  The Fabric of Real ThingsStuff That Makes Stuff Happen

Previous meditation: "Does your rule forbid epiphenomenalist theories of consciousness that consciousness is caused by neurons, but doesn't affect those neurons in turn? The classic argument for epiphenomenal consciousness is that we can imagine a universe where people behave exactly the same way, but there's nobody home - no awareness, no consciousness, inside the brain. For all the atoms in this universe to be in the same place - for there to be no detectable difference internally, not just externally - 'consciousness' would have to be something created by the atoms in the brain, but which didn't affect those atoms in turn. It would be an effect of atoms, but not a cause of atoms. Now, I'm not so much interested in whether you think epiphenomenal theories of consciousness are true or false - rather, I want to know if you think they're impossible or meaningless a priori based on your rules."

Is it coherent to imagine a universe in which a real entity can be an effect but not a cause?

Well... there's a couple of senses in which it seems imaginable. It's important to remember that imagining things yields info primarily about what human brains can imagine. It only provides info about reality to the extent that we think imagination and reality are systematically correlated for some reason.

That said, I can certainly write a computer program in which there's a tier of objects affecting each other, and a second tier - a lower tier - of epiphenomenal objects which are affected by them, but don't affect them. For example, I could write a program to simulate some balls that bounce off each other, and then some little shadows that follow the balls around.

But then I only know about the shadows because I'm outside that whole universe, looking in. So my mind is being affected by both the balls and shadows - to observe something is to be affected by it. I know where the shadow is, because the shadow makes pixels be drawn on screen, which make my eye see pixels. If your universe has two tiers of causality - a tier with things that affect each other, and another tier of things that are affected by the first tier without affecting them - then could you know that fact from inside that universe?

Again, this seems easy to imagine as long as objects in the second tier can affect each other. You'd just have to be living in the second tier! We can imagine, for example - this wasn't the way things worked out in our universe, but it might've seemed plausible to the ancient Greeks - that the stars in heaven (and the Sun as a special case) could affect each other and affect Earthly forces, but no Earthly force could affect them:

(Here the X'd-arrow stands for 'cannot affect'.)

The Sun's light would illuminate Earth, so it would cause plant growth. And sometimes you would see two stars crash into each other and explode, so you'd see they could affect each other. (And affect your brain, which was seeing them.) But the stars and Sun would be made out of a different substance, the 'heavenly material', and throwing any Earthly material at it would not cause it to change state in the slightest. The Earthly material might be burned up, but the Sun would occupy exactly the same position as before. It would affect us, but not be affected by us.

(To clarify an important point raised in the comments: In standard causal diagrams and in standard physics, no two individual events ever affect each other; there's a causal arrow from the PAST to FUTURE but never an arrow from FUTURE to PAST. What we're talking about here is the sun and stars over time, and the generalization over causal arrows that point from Star-in-Past to Sun-in-Present and Sun-in-Present back to Star-in-Future. The standard formalism dealing with this would be Dynamic Bayesian Networks (DBNs) in which there are repeating nodes and repeating arrows for each successive timeframe: X1, X2, X3, and causal laws F relating Xi to Xi+1. If the laws of physics did not repeat over time, it would be rather hard to learn about the universe! The Sun repeatedly sends out photons, and they obey the same laws each time they fall on Earth; rather than the Fi being new transition tables each time, we see a constant Fphysics over and over. By saying that we live in a single-tier universe, we're observing that whenever there are F-arrows, causal-link-types, which (over repeating time) descend from variables-of-type-X to variables-of-type-Y (like present photons affecting future electrons), there are also arrows going back from Ys to Xs (like present electrons affecting future photons). If we weren't generalizing over time, it couldn't possibly make sense to speak of thingies that "affect each other" - causal diagrams don't allow directed cycles!)

A two-tier causal universe seems easy to imagine, even easy to specify as a computer program. If you were arranging a Dynamic Bayes Net at random, would it randomly have everything in a single tier? If you were designing a causal universe at random, wouldn't there randomly be some things that appeared to us as causes but not effects? And yet our own physicists haven't discovered any upper-tier particles which can move us without being movable by us. There might be a hint here at what sort of thingies tend to be real in the first place - that, for whatever reasons, the Real Rules somehow mandate or suggest that all the causal forces in a universe be on the same level, capable of both affecting and being affected by each other.

Still, we don't actually know the Real Rules are like that; and so it seems premature to assign a priori zero probability to hypotheses with multi-tiered causal universes. Discovering a class of upper-tier affect-only particles seems imaginable[1] - we can imagine which experiences would convince us that they existed. If we're in the Matrix, we can see how to program a Matrix like that. If there's some deeper reason why that's impossible in any base-level reality, we don't know it yet. So we probably want to call that a meaningful hypothesis for now.

But what about lower-tier particles which can be affected by us, and yet never affect us?

Perhaps there are whole sentient Shadow Civilizations living on my nose hairs which can never affect those nose hairs, but find my nose hairs solid beneath their feet. (The solid Earth affecting them but not being affected, like the Sun's light affecting us in the 'heavenly material' hypothesis.) Perhaps I wreck their world every time I sneeze. It certainly seems imaginable - you could write a computer program simulating physics like that, given sufficient perverseness and computing power...

And yet the fundamental question of rationality - "What do you think you know, and how do you think you know it?" - raises the question:

How could you possibly know about the lower tier, even if it existed?

To observe something is to be affected by it - to have your brain and beliefs take on different states, depending on that thing's state. How can you know about something that doesn't affect your brain?

In fact there's an even deeper question, "How could you possibly talk about that lower tier of causality even if it existed?"

Let's say you're a Lord of the Matrix. You write a computer program which first computes the physical universe as we know it (or a discrete approximation), and then you add a couple of lower-tier effects as follows:

First, every time I sneeze, the binary variable YES_SNEEZE will be set to the second of its two possible values.

Second, every time I sneeze, the binary variable NO_SNEEZE will be set to the first of its two possible values.

Now let's say that - somehow - even though I've never caught any hint of the Matrix - I just magically think to myself one day, "What if there's a variable that watches when I sneeze, and gets set to 1?"

It will be all too easy for me to imagine that this belief is meaningful and could be true or false:

And yet in reality - as you know from outside the matrix - there are two shadow variables that get set when I sneeze. How can I talk about one of them, rather than the other? Why should my thought about '1' refer to their second possible value rather than their first possible value, inside the Matrix computer program? If we tried to establish a truth-value in this situation, to compare my thought to the reality inside the computer program - why compare my thought about SNEEZE_VAR to the variable YES_SNEEZE instead of NO_SNEEZE, or compare my thought '1' to the first possible value instead of the second possible value?

Under more epistemically healthy circumstances, when you talk about things that are not directly sensory experiences, you will reference a causal model of the universe that you inducted to explain your sensory experiences. Let's say you repeatedly go outside at various times of day, and your eyes and skin directly experience BRIGHT-WARM, BRIGHT-WARM, BRIGHT-WARM, DARK-COOL, DARK-COOL, etc. To explain the patterns in your sensory experiences, you hypothesize a latent variable we'll call 'Sun', with some kind of state which can change between 1, which causes BRIGHTness and WARMness, and 0, which causes DARKness and COOLness. You believe that the state of the 'Sun' variable changes over time, but usually changes less frequently than you go outside.

p(BRIGHT|Sun=1) 0.9
p(¬BRIGHT|Sun=1) 0.1
p(BRIGHT|Sun=0) 0.1
p(¬BRIGHT|Sun=0) 0.9

Standing here outside the Matrix, we might be tempted to compare your beliefs about "Sun = 1", to the real universe's state regarding the visibility of the sun in the sky (or rather, the Earth's rotational position).

But even if we compress the sun's visibility down to a binary categorization, how are we to know that your thought "Sun = 1" is meant to correspond to the sun being visible in the sky, rather than the sun being occluded by the Earth? Why the first state of the variable, rather than the second state?

How indeed are we know that this thought "Sun = 1" is meant to compare to the sun at all, rather than an anteater in Venezuela?

Well, because that 'Sun' thingy is supposed to be the cause of BRIGHT and WARM feelings, and if you trace back the cause of those sensory experiences in reality you'll arrive at the sun that the 'Sun' thought allegedly corresponds to. And to distinguish between whether the sun being visible in the sky is meant to correspond to 'Sun'=1 or 'Sun'=0, you check the conditional probabilities for that 'Sun'-state giving rise to BRIGHT - if the actual sun being visible has a 95% chance of causing the BRIGHT sensory feeling, then that true state of the sun is intended to correspond to the hypothetical 'Sun'=1, not 'Sun'=0.

Or to put it more generally, in cases where we have...

...then the correspondence between map and territory can at least in principle be point-wise evaluated by tracing causal links back from sensory experiences to reality, and tracing hypothetical causal links from sensory experiences back to hypothetical reality. We can't directly evaluate that truth-condition inside our own thoughts; but we can perform experiments and be corrected by them.

Being able to imagine that your thoughts are meaningful and that a correspondence between map and territory is being maintained, is no guarantee that your thoughts are true. On the other hand, if you can't even imagine within your own model how a piece of your map could have a traceable correspondence to the territory, that is a very bad sign for the belief being meaningful, let alone true. Checking to see whether you can imagine a belief being meaningful is a test which will occasionally throw out bad beliefs, though it is no guarantee of a belief being good.


Okay, but what about the idea that it should be meaningful to talk about whether or not a spaceship continues to exist after it travels over the cosmological horizon? Doesn't this theory of meaningfulness seem to claim that you can only sensibly imagine something that makes a difference to your sensory experiences?

No. It says that you can only talk about events that your sensory experiences pin down within the causal graph. If you observe enough protons, electrons, neutrons, and so on, you can pin down the physical generalization which says, "Mass-energy is neither created nor destroyed; and in particular, particles don't vanish into nothingness without a trace." It is then an effect of that rule, combined with our previous observation of the ship itself, which tells us that there's a ship that went over the cosmological horizon and now we can't see it any more.

To navigate referentially to the fact that the ship continues to exist over the cosmological horizon, we navigate from our sensory experience up to the laws of physics, by talking about the cause of electrons not blinking out of existence; we also navigate up to the ship's existence by tracing back the cause of our observation of the ship being built. We can't see the future ship over the horizon - but the causal links down from the ship's construction, and from the laws of physics saying it doesn't disappear, are both pinned down by observation - there's no difficulty in figuring out which causes we're talking about, or what effects they have.[2]


All righty-ighty, let's revisit that meditation:

"Does your rule forbid epiphenomenalist theories of consciousness in which consciousness is caused by neurons, but doesn't affect those neurons in turn? The classic argument for epiphenomenal consciousness is that we can imagine a universe where people behave exactly the same way, but there's nobody home - no awareness, no consciousness, inside the brain. For all the atoms in this universe to be in the same place - for there to be no detectable difference internally, not just externally - 'consciousness' would have to be something created by the atoms in the brain, but which didn't affect those atoms in turn. It would be an effect of atoms, but not a cause of atoms. Now, I'm not so much interested in whether you think epiphenomenal theories of consciousness are true or false - rather, I want to know if you think they're impossible or meaningless a priori based on your rules."

The closest theory to this which definitely does seem coherent - i.e., it's imaginable that it has a pinpointed meaning - would be if there was another little brain living inside my brain, made of shadow particles which could affect each other and be affected by my brain, but not affect my brain in turn. This brain would correctly hypothesize the reasons for its sensory experiences - that there was, from its perspective, an upper tier of particles interacting with each other that it couldn't affect. Upper-tier particles are observable, i.e., can affect lower-tier senses, so it would be possible to correctly induct a simplest explanation for them. And this inner brain would think, "I can imagine a Zombie Universe in which I am missing, but all the upper-tier particles go on interacting with each other as before." If we imagine that the upper-tier brain is just a robotic sort of agent, or a kitten, then the inner brain might justifiably imagine that the Zombie Universe would contain nobody to listen - no lower-tier brains to watch and be aware of events.

We could write that computer program, given significantly more knowledge and vastly more computing power and zero ethics.

But this inner brain composed of lower-tier shadow particles cannot write upper-tier philosophy papers about the Zombie universe. If the inner brain thinks, "I am aware of my own awareness", the upper-tier lips cannot move and say aloud, "I am aware of my own awareness" a few seconds later. That would require causal links from lower particles to upper particles.

If we try to suppose that the lower tier isn't a complicated brain with an independent reasoning process that can imagine its own hypotheses, but just some shadowy pure experiences that don't affect anything in the upper tier, then clearly the upper-tier brain must be thinking meaningless gibberish when the upper-tier lips say, "I have a lower tier of shadowy pure experiences which did not affect in any way how I said these words." The deliberating upper brain that invents hypotheses for sense data, can only use sense data that affects the upper neurons carrying out the search for hypotheses that can be reported by the lips. Any shadowy pure experiences couldn't be inputs into the hypothesis-inventing cognitive process. So the upper brain would be talking nonsense.

There's a version of this theory in which the part of our brain that we can report out loud, which invents hypotheses to explain sense data out loud and manifests physically visible papers about Zombie universes, has for no explained reason invented a meaningless theory of shadow experiences which is experienced by the shadow part as a meaningful and correct theory.  So that if we look at the "merely physical" slice of our universe, philosophy papers about consciousness are meaningless and the physical part of the philosopher is saying things their physical brain couldn't possibly know even if they were true.  And yet our inner experience of those philosophy papers is meaningful and true. In a way that couldn't possibly have caused me to physically write the previous sentence, mind you. And yet your experience of that sentence is also true even though, in the upper tier of the universe where that sentence was actually written, it is not only false but meaningless.

I'm honestly not sure what to say when a conversation gets to that point. Mostly you just want to yell, "Oh, for the love of Belldandy, will you just give up already?" or something about the importance of saying oops.

(Oh, plus the unexplained correlation violates the Markov condition for causal models.)

Maybe my reply would be something along the lines of, "Okay... look... I've given my account of a single-tier universe in which agents can invent meaningful explanations for sense data, and when they build accurate maps of reality there's a known reason for the correspondence... if you want to claim that a different kind of meaningfulness can hold within a different kind of agent divided into upper and lower tiers, it's up to you to explain what parts of the agent are doing which kinds of hypothesizing and how those hypotheses end up being meaningful and what causally explains their miraculous accuracy so that this all makes sense."

But frankly, I think people would be wiser to just give up trying to write sensible philosophy papers about lower causal tiers of the universe that don't affect the philosophy papers in any way.


MeditationIf we can only meaningfully talk about parts of the universe that can be pinned down inside the causal graph, where do we find the fact that 2 + 2 = 4? Or did I just make a meaningless noise, there? Or if you claim that "2 + 2 = 4" isn't meaningful or true, then what alternate property does the sentence "2 + 2 = 4" have which makes it so much more useful than the sentence "2 + 2 = 3"?


Mainstream status.


 [1] Well, it seems imaginable so long as you toss most of quantum physics out the window and put us back in a classical universe. For particles to not be affected by us, they'd need their own configuration space such that "which configurations are identical" was determined by looking only at those particles, and not looking at any lower-tier particles entangled with them. If you don't want to toss QM out the window, it's actually pretty hard to imagine what an upper-tier particle would look like.

 [2] This diagram treats the laws of physics as being just another node, which is a convenient shorthand, but probably not a good way to draw the graph. The laws of physics really correspond to the causal arrows Fi, not the causal nodes Xi. If you had the laws themselves - the function from past to future - be an Xi of variable state, then you'd need meta-physics to describe the Fphysics arrows for how the physics-stuff Xphysics could affect us, followed promptly by a need for meta-meta-physics et cetera. If the laws of physics were a kind of causal stuff, they'd be an upper tier of causality - we can't appear to affect the laws of physics, but if you call them causes, they can affect us. In Matrix terms, this would correspond to our universe running on a computer that stored the laws of physics in one area of RAM and the state of the universe in another area of RAM, the first area would be an upper causal tier and the second area would be a lower causal tier. But the infinite regress from treating the laws of determination as causal stuff, makes me suspicious that it might be an error to treat the laws of physics as "stuff that makes stuff happen and happens because of other stuff". When we trust that the ship doesn't disappear when it goes over the horizon, we may not be navigating to a physics-node in the graph, so much as we're navigating to a single Fphysics that appears in many different places inside the graph, and whose previously unknown function we have inferred. But this is an unimportant technical quibble on Tuesdays, Thursdays, Saturdays, and Sundays. It is only an incredibly deep question about the nature of reality on Mondays, Wednesdays, and Fridays, i.e., less than half the time.

Part of the sequence Highly Advanced Epistemology 101 for Beginners

Next post: "Proofs, Implications, and Models"

Previous post: "Stuff That Makes Stuff Happen"

Comments (242)

Comment author: CronoDAS 22 October 2012 12:23:50AM 18 points [-]

Epiphenominal theories of consciousness are kind of silly, but here's another situation I can wonder about... some cellular automata rules, including the Turing-complete Conway's Game of Life, can have different "pasts" that can lead to the same present. From the point of view of a being living in such a universe (one in which information can be destroyed), is there a fact of the matter as to which "past" actually happened?

Comment author: Pentashagon 22 October 2012 11:16:56PM 8 points [-]

I had always thought that our physical universe had this property as well, i.e. the Everett multiverse branches into the past as well as into the future.

Comment author: Eliezer_Yudkowsky 25 October 2012 08:01:33AM 11 points [-]

If you take a single branch and run it backward, you'll find that it diverges into a multiverse of its own. If you take all the branches and run them backward, their branches will cohere instead of decohering, cancel out in most places, and miraculously produce only the larger, more coherent blobs of amplitude they started from. Sort of like watching an egg unscramble itself.

Comment author: SilasBarta 28 October 2012 12:06:52AM 1 point [-]

If you take all the branches and run them backward, their branches will cohere instead of decohering, cancel out in most places, and miraculously produce only the larger, more coherent blobs of amplitude they started from.

And the beings in them will only have memories of further-cohered (further "pastward") events, just as if you didn't run anything backwards.

Comment author: Viliam_Bur 25 October 2012 07:40:09AM *  1 point [-]

And at the beginning of the universe we have a set of states which just point time-backwards at each other, which is why we cannot go meaningfully more backwards in time.

Something like:
A1 goes with probability 1% to B1, 1% to C1, and 98% to A2.
B1 goes with probability 1% to A1, 1% to C1, and 98% to B2.
C1 goes with probability 1% to A1, 1% to B1, and 98% to C2.

So if you ask about the past of A2, you get A1, which is the part that makes intuitive sense for us. But trying to go deeper in the past just gives us that the past of A1 is B1 or C1, and the past of B1 is A1 or C1, etc. Except that the change does not clearly happen in one moment (A2 has a rather well-defined past, A1 does not), but more gradually.

Comment author: Eliezer_Yudkowsky 25 October 2012 08:00:07AM 1 point [-]

As I understand it, this is not how standard physics models the beginning of time.

Comment author: [deleted] 25 October 2012 06:26:35PM 2 points [-]

I don't think anyone takes seriously the way standard physics models the beginning of time (temperature and density of the universe approaching infinity as its age approaches zero), anyway, as it's most likely incorrect due to quantum gravity effects.

Comment author: wedrifid 25 October 2012 10:21:14PM 0 points [-]

I don't think anyone takes seriously the way standard physics models the beginning of time (temperature and density of the universe approaching infinity

This is a correct usage of terminology but the irony still made me smile.

Comment author: [deleted] 26 October 2012 12:35:45AM 1 point [-]

What?

Comment author: RobbBB 09 June 2013 11:10:18PM 1 point [-]

I think wedrifid is pointing to the irony in saying that the 'standard' model is (on some issue) standardly rejected.

Comment author: Viliam_Bur 25 October 2012 09:24:06AM 0 points [-]

Oh. I tried to find something, but the only thing that partially pattern-matches it was the Hartle–Hawking state. If we mix it with the "universe as a Markov chain over particle configurations" model, it could lead to something like this. Or could not.

Comment author: Eliezer_Yudkowsky 22 October 2012 12:55:49AM 4 points [-]

Interesting question! I'd say that you could refer to the possibilities as possibilities, e.g. in a debate over whether a particular past would in fact have led to the present, but to speak of the 'actual past' might make no sense because you couldn't get there from there... no, actually, I take that back, you might be able to get there via simplicity. I.e. if there's only one past that would have evolved from a simply-tiled start state for the automaton.

Comment author: CCC 25 October 2012 08:35:49AM 2 points [-]

In the Game of Life, a single live cell with no neighbours will become a dead cell in the next step. Therefore, any possible present state that has at least one past state has an infinite number of one-step-back states (which differ from the one state merely in having one or more neighbourless cells at random locations, far enough from anything else to have no effect).

Some of these one-step-back states may end up having evolved from simpler starting tilesets than the one with no vanishing cells.

Comment author: ialdabaoth 22 October 2012 11:08:41PM *  3 points [-]

But does it really matter? If both states are possible, why not just say "my past contains ambiguity?"

With quantum mechanics, even though the "future" itself (as a unified wavefunction) evolves forward as a whole, the bit-that-makes-up-this-pseudofactor-of-me has multiple possible outcomes. We live with future ambiguity just fine, and quantum mechanics forces us to say "both experienced futures must be dealt with probabilistically". Even though the mechanism is different, what's wrong with treating the "past" as containing the same level of branching as the future?

EDIT: From a purely global, causal perspective, I understand the desire to be able to say, "both X and Y can directly cause Z, but in point of fact, this time it was Y." But you're inside, so you don't get to operate as a thing that can distinguish between X and Y, and this isn't necessarily an "orbital teapot" level of implausibility. If configuration Y is 10^4 more likely as a 'starting' configuration than configuration X according to your understanding of how starting configurations are chosen, then sure - go ahead and assert that it was (or may-as-well-have-been) configuration Y that was your "actual" past - but if the configuration probabilities are more like 70%/30%, or if your confidence that you understand how starting configurations are chosen is low enough, then it may be better to just swallow the ambiguity.

EDIT2: Coming from a completely different angle, why assert that one or the other "happened", rather than looking at it as a kind of path-integral? It's a celular automaton, instead of a quantum wave-function, which means that you're summing discrete paths instead of integrating infinitesimals, but it seems (at first glance) that the reasoning is equally applicable.

Comment author: Viliam_Bur 25 October 2012 07:59:16AM *  1 point [-]

If both states are possible, why not just say "my past contains ambiguity?"

Ambiguity it is, but we usually want to know the probabilities. If I tell you that whether you win or not win a lottery tomorrow is "ambiguous", you would not be satisfied with such answer, and you would ask how much likely it is to win. And this question somehow makes sense even if the lottery is decided by a quantum event, so you know that each future happens in some Everett branch.

Similarly, in addition to knowing that the past is ambiguous, we should ask how likely are the individual pasts. In our universe you would want to know how the pasts P1 and P2 are likely to become NOW. The Conway's Game of Life does not branch time-forward, so if you have two valid pasts, their probabilities of becoming NOW are 100% each.

But that is only a part of the equation. The other part are the prior probabilities of P1 and P2. Even if both P1 and P2 deterministically evolve to NOW, their prior probabilities influence how likely did NOW really evolve from each of them.

I am not sure what would be the equivalent of Solomonoff induction for the Conway's Game of Life. Starting with a finite number of "on" cells, where each additional "on" cell decreases the prior probability of the configuration? Starting with an infinite plane where each cell has a 50% probability to be "on"? Or an infinite plane with each cell having a p probability of being "on", where p has the property that after one step in such plane, the average ratio of "on" cells remain the same (the p being kind-of-eigenvalue of the rules)?

But the general idea is that if P1 is somehow "generally more likely to happen" than P2, we should consider P1 to be more likely the past of NOW than P2, even if both P1 and P2 deterministically evolve to NOW.

Comment author: Pentashagon 22 October 2012 11:14:36PM 1 point [-]

no, actually, I take that back, you might be able to get there via simplicity. I.e. if there's only one past that would have evolved from a simply-tiled start state for the automaton.

The simplest start state might actually be a program that simulates the evolution of every possible starting state in parallel. If time and space are unbounded and an entity is more complex than the shortest such program then it is more likely that the entity is the result of the program and not the result of evolving from another random state.

Comment author: mfb 25 October 2012 03:10:45PM 1 point [-]

The Standard Model of particle physics with MWI is time-symmetric (to be precise: CPT symmetric) and conserves information. If you define the precise state at one point in time, you can calculate the unique past which lead to that state and the unique future which will evolve from that state. Note that for general states, "past" and "future" are arbitrary definitions.

Comment author: CronoDAS 25 October 2012 04:44:31PM 3 points [-]

(Which is why I specified a different set of laws of physics.)

Comment author: abramdemski 16 November 2012 05:21:05AM 1 point [-]

I am unable to see the appeal of a view in which there is no fact of the matter. It seems to me that there is a fact of the matter concerning the past, even if it is impossible for us to know. This is not similar to the case where sneezing alters two shadow variables, and it is impossible for us to meaningfully refer to variable 1 as opposed to variable 2; the past has a structure, so assertions will typically have definite referents.

Comment author: hwc 22 October 2012 09:06:24PM 1 point [-]

Or, a Boltzmann brain that flickered into existence with memories of a past that never happened.

Comment author: ialdabaoth 23 October 2012 12:54:18AM 2 points [-]

In that particular case, "never happened" has some weird ontological baggage. If a simulated consciousness is still conscious, then isn't its simulated past still a past?

Perhaps "didn't happen" in the sense that its future reality will not conform to its memory-informed expectations, but it seems like, if those memories form a coherent 'past', then in a simulationist sense that past did happen, even if it wasn't simulated with perfect fidelity.

Comment author: drethelin 22 October 2012 09:20:32PM 0 points [-]

This is actually one of the reasons I have to doubt Cryonics. You can talk about nano-tech being able to "reverse" the damage, but it's possible (and I think likely), that it's very hard to go from damaged states to the specific non-damaged state that actually constitutes your consciousness/memory.

Comment author: ialdabaoth 22 October 2012 11:13:28PM 3 points [-]

Assuming that "you" are a point in consciousness phase-space, and not a "smear". If "you-ness" is a locus of similar-but-slightly-different potential states, then "mostly right" is going to be good enough.

And, given that every morning when you wake up, you're different-but-still-you, I'd say that there's strong evidence that "you-ness" is a locus of similar-but-slightly-different potential states, rather than a singular point.

This means, incidentally, that it may be possible to resurrect people without a physical copy of their brains at all, if enough people who remember them well enough when the technology becomes available.

Of course, since it's a smear, the question becomes "where do you want to draw the line between Bob and not-Bob?" - since whatever you create will believe it's Bob, and will act the way everyone alive remembers that Bob acted, and the "original" isn't around to argue (assuming you believe in concepts like "original" to begin with, but if you do, you have some weirder paradoxes to deal with).

Comment author: DaFranker 22 October 2012 09:36:00PM 1 point [-]

Which is why it's better for there to be more people signed up, but not actually being frozen yet. The more money they get while the later you get frozen, the better the odds. If immortality is something you want, this still seems like the best gamble.

Comment author: Armok_GoB 21 October 2012 04:25:36PM 14 points [-]

Just for the Least Convenient World, what if the zombies build a supercomputer and simulate random universes, and find that in 98% of simulated universes life forms like theirs do have shadow brains, and that the programs for the remaining 2% are usually significantly longer?

Comment author: Liron 23 October 2012 04:47:23AM 0 points [-]

How can the version without shadow brains be significantly longer? Even in the worst possible world, it seems like the 2% of non-shadow-brain programs could be encoded by copying their corresponding shadow-brain programs and adding a few lines telling the computer how to garbage-collect shadows using a straightforward pruning algorithm on the causal graph.

Comment author: Armok_GoB 23 October 2012 04:45:10PM 2 points [-]

by the programs being short enough in the first place that those few lines still doubles the length? By the universe like part not being straightforwardly encoded so that to distinguish anything about it you first need a long AI-like interpreter just to get there?

Comment author: shminux 21 October 2012 05:08:22PM 9 points [-]

Is it coherent to imagine a universe in which a real entity can be an effect but not a cause?

Your favorite example of event horizons, cosmological or otherwise, is like that. GR suggests that there can be a ring singularity inside an eternal spinning black hole (but not one spun up from rest), near/around which you can go forever without being crushed. (it also suggests that there could be closed timelike curves around it, but I'll ignore this for now.) So maybe there are particles/objects/entities living there.

Stuff thrown into such a black hole can certainly affect the hypothetical entities living inside. Like a meteor shower from the outside. But the outside is not affected by anything happening inside, the horizon prevents it.

Comment author: AnthonyC 22 October 2012 01:23:22AM 1 point [-]

Fair, but quantum mechanics gives us Hawking radiation, which may or may not provide information (in principle) about what went into the black hole.

Also, there are causal arrows from the black hole to everything it pulls on, and those are ultimately the sum of causal arrows from each particle in the black hole even if from the outside we can't discern the individual particles.

Comment author: MrMind 22 October 2012 02:54:30PM 0 points [-]

Stuff thrown into such a black hole can certainly affect the hypothetical entities living inside. Like a meteor shower from the outside. But the outside is not affected by anything happening inside, the horizon prevents it.

That is not entirely true: stuff thrown into the black hole increases the horizon area and possibly modifies its geometry, and in return the horizon affects the spatial infinity (the area around the horizon). The debate is about how much information the horizon deletes in the process. The same is for the cosmological horizon, which is effectively just another kind of singularity.

Comment author: shminux 22 October 2012 02:57:07PM 2 points [-]

stuff thrown into the black hole increases the horizon area and possibly modifies its geometry, and in return the horizon affects the spatial infinity (the area around the horizon).

That's outside affecting outside, not inside affecting outside.

The same is for the cosmological horizon, which is effectively just another kind of singularity.

Horizon is not a singularity.

Comment author: MrMind 22 October 2012 03:52:56PM *  0 points [-]

That's outside affecting outside, not inside affecting outside.

Hmm... let's taboo "outside" and "inside". The properties of stuff within the horizon affect the properties of the horizon, which in turn affect the properties of space-matter at spatial infinity. Is this formulation more acceptable?

Horizon is not a singularity.

Right, I'll rephrase: the same goes for the cosmological horizon, which effectively 'surrounds' just another kind of singularity.

Comment author: shminux 22 October 2012 04:22:24PM 3 points [-]

The properties of stuff within the horizon affect the properties of the horizon

Wrong. There could be tons of different things going on inside, absolutely indistinguishable from outside, which only sees mass, electric charge and angular momentum. There is no causal connection from inside to outside whatsoever, barring FTL communication.

Right, I'll rephrase: the same goes for the cosmological horizon, which effectively 'surrounds' just another kind of singularity.

Wrong again. There is no singularity of any kind behind the cosmological horizon (which is not a closed surface to begin with, so it cannot "surround" anything). Well, there might be black holes and stuff, or there might not be, but there is certainly not a requirement of anything singular being there. Consider googling the definition of singularity in general relativity.

Comment author: DaFranker 22 October 2012 05:03:55PM *  0 points [-]

Wrong. There could be tons of different things going on inside, absolutely indistinguishable from outside, which only sees mass, electric charge and angular momentum. There is no causal connection from inside to outside whatsoever, barring FTL communication.

Unless the "inside" was spontaneously materialized into existence while simultaneously a different chunk of the singularity's mass blinked out of existence in manners which defy nearly all the physics I know, then there still remains a causal connection from the "disappearance" of this stuff that's "inside" from the world "outside" at some point in outside time frames, AFAICT. This disappearance of specific pieces of matter and energy seems to more than qualify as a causal effect, when compared to counterfactual futures where they do not disappear.

Also, the causal connection [Inside -> Mass -> Outside] pretty much looks like a causal connection from inside to outside to me. There's this nasty step in the middle that blurs all the information such that under most conceivable circumstances there's no way to tell which of all possible insides is the "true" one, but combined with the above about matter disappearance can still let you concentrate your probability mass, compared to meaningless epiphenomena that cover the entire infinite hypothesis space (minus one single-dimensional line representing its interaction with anything that interacts with our reality in any way) with equal probability because there's no way it could even in principle affect us even in CTCs, FTL, timeless or n-dimensional spaces, etc.

(Note: I'm not an expert on mind-bending hypothetical edge cases of theoretical physics, so I'm partially testing my own understanding of the subject here.)

Comment author: shminux 22 October 2012 05:57:03PM *  0 points [-]

I'm partially testing my own understanding of the subject here.

Most of what you said is either wrong or meaningless, so I don't know where to begin unraveling it, sorry. Feel free to ask simple questions of limited scope if you want to learn more about black holes, horizons, singularities and related matters. The subject is quite non-trivial and often counter-intuitive.

Comment author: DaFranker 22 October 2012 07:56:54PM 0 points [-]

Hmm, alright.

In more vague, amateur terms, isn't the whole horizon thing always the same case, i.e. it's causally linked to the rest of the universe by observations in the past and inferences using presumed laws of physics, even if the actual state of things beyond the horizon (or inside it or whatever) doesn't change what we can observe?

Comment author: shminux 22 October 2012 08:19:22PM 1 point [-]

The event horizon in an asymptotically flat spacetime (which is not quite the universe we live in, but a decent first step) is defined as the causal past of the infinite causal future. This definition guarantees that we see no effects whatsoever from the part of the universe that is behind the event horizon. The problem with this definition is that we have to wait forever to draw the horizon accurately. Thus there are several alternative horizons which are more instrumentally useful for theorem proving and/or numerical simulations, but are not in general identical to the event horizon. The cosmological event horizon is a totally different beast (it is similar to the Rindler horizon, used to derive the Unruh effect), though it does share a number of properties with the black hole event horizon. There are further exciting complications once you get deeper into the subject.

Comment author: Alejandro1 22 October 2012 04:35:54PM 0 points [-]

Wrong. There could be tons of different things going on inside, absolutely indistinguishable from outside, which only sees mass, electric charge and angular momentum.

Nitpick: this is only true for a stationary black hole. If you throw something sufficiently big in, you would expect the shape of the horizon to change and bulge a bit, until it settles down into a stationary state for a larger black hole. You are of course correct that this does not allow anything inside to send a signal to the outside.

Comment author: shminux 22 October 2012 04:41:07PM *  2 points [-]

Nitpick: this is only true for a stationary black hole.

Right, I didn't want to go into these details, MrMind seems confused enough as it is. I'd have to explain that the horizon shape is only determined by what falls in, and eventually talk about apparent and dynamical horizons and marginally outer trapped surfaces...

Comment author: MrMind 24 October 2012 02:33:07PM 0 points [-]

Wrong. There could be tons of different things going on inside, absolutely indistinguishable from outside, which only sees mass, electric charge and angular momentum.

Also entropy. Anyway, those are determined by the mass, electrical charge and angular momentum of the matter that fell inside. We may not want to call it a causal connection, but it's certainly a case of properties within determining properties outside.

There is no causal connection from inside to outside whatsoever, barring FTL communication.

There is no direct causal connection, meaning a worldline from the inside to the outside of the black hole. But even if the horizon screens almost all of the infalling matter properties, it doesn't screen everything (and probably, but this is a matter of quantum gravity, doesn't screen nothing).

Wrong again. There is no singularity of any kind behind the cosmological horizon (which is not a closed surface to begin with, so it cannot "surround" anything). Well, there might be black holes and stuff, or there might not be, but there is certainly not a requirement of anything singular being there. Consider googling the definition of singularity in general relativity.

I'll admit to not have much knowledge about this specific theme, and I'll educate myself more properly, but in the case of my earlier sentence I used "singularity" as a mathematical term, referring to a region of spacetime in which the GR equations acquire a singular value, so not specifically to a gravitational singularity like a black-hole or a domain wall. In the case of most commonplace cosmological horizons, this region is simply space-like infinity.

Comment author: betterthanwell 21 October 2012 03:21:36AM 5 points [-]

Mainstream status points to /Eliezer_Yudkowsky-drafts/ (Forbidden: You aren't allowed to do that.)

Comment author: beoShaffer 21 October 2012 04:47:12AM 0 points [-]

So does the Mediation link.

Comment author: Eliezer_Yudkowsky 20 October 2012 08:47:54AM 5 points [-]

Mainstream status:

I haven't yet happened to run across a philosophical position which says that meaningful correspondences between hypotheses and reality can only be pinned down by following Pearl-style causal links inferred as the simplest explanation of observed experiences, and that only this can allow an agent to consistently believe that its beliefs are meaningful.

In fact, I haven't seen anything at all about referential meaningfulness requiring cause-and-effect links with the phenomenon, just like I haven't seen anything about a universe being a connected fabric of causes and effects; but it wouldn't surprise me if either of those ideas were out there somewhere.

(There's a "causal theory of reference" listed in Wikipedia but it doesn't seem to be about remotely the same subject matter; the theory's tenets seem to be that "a name's referent is fixed by an original act of naming", and that "later uses of the name succeed in referring to the referent by being linked to that original act via a causal chain".)

EDIT: Apparently causal theories of reference have been used to argue against Zombie Worlds so I stand corrected on this point. See below.

Comment author: pragmatist 21 October 2012 06:39:05PM *  10 points [-]

As bryjnar points out, all the stuff you say here (subtracting out the Pearl stuff) is entailed by the causal theory of reference. The reason quick summaries of that view will seem unfamiliar is that most of the early work on the causal theory was primarily motivated by a different concern -- accounting for how our words acquire their meaning. Thus the focus on causal chains from "original acts of naming" and whatnot. However, your arguments against epiphenomenalism all hold in the causal theory.

It is true that nobody (that I know of) has developed an explicitly Pearlian causal theory of reference, but this is really accounted for by division of labor in philosophy. People working on reference will develop a causal theory of reference and use words like "cause" without specifying what they mean by it. If you ask them what they mean, they will say "Whatever the best theory of causation is. Go ask the people working on causation about that." And among the people working on causation, there are indeed philosophers who have built on Pearlian ideas. Christopher Hitchcock and James Woodward, for instance.

Comment author: Eliezer_Yudkowsky 21 October 2012 09:23:12PM 2 points [-]

However, your arguments against epiphenomenalism all hold in the causal theory.

Has anyone made them? I ask because every use I've seen of the 'causal theory of reference' is, indeed, about "accounting for how our words acquire their meaning", something of a nonproblem from the standpoint of somebody who thinks that words don't have inherent meanings.

Comment author: pragmatist 21 October 2012 09:44:48PM 19 points [-]

The issue is broached by Chalmers himself in The Conscious Mind (p. 201). He says:

... it is sometimes said that reference to an entity requires a causal connection to that entity; this is known as the causal theory of reference. If so, then it would be impossible to refer to causally irrelevant experiences.

He goes on to reject the causal theory of reference.

Here is a relevant excerpt from the SEP article on zombies:

But, arguably, it is a priori true that phenomenal consciousness, whether actual or possible, involves being able to refer to and know about one's qualia. If that is right, any zombie-friendly account faces a problem. According to the widely accepted causal theory of reference — accepted by many philosophers — reference and knowledge require us to be causally affected by what is known or referred to (Kripke 1972/80); and it seems reasonable to suppose that this too is true a priori if true at all. On that basis, in those epiphenomenalistic worlds whose conceivability seems to follow from the conceivability of zombies — (worlds where qualia are inert) — our counterparts cannot know about or refer to their qualia. That contradicts the assumption that phenomenal consciousness involves being able to refer to qualia, from which it follows that such epiphenomenalistic worlds are not possible after all. Hence zombies are not conceivable in the relevant sense either, since their conceivability leads a priori to a contradiction. To summarize: if zombies are conceivable, so are epiphenomenalistic worlds. But by the causal theory of reference, epiphenomenalistic worlds are not conceivable; therefore zombies are not conceivable.

Comment author: Eliezer_Yudkowsky 21 October 2012 10:24:38PM 6 points [-]

Thanks! I stand corrected.

Comment author: bryjnar 21 October 2012 10:44:31AM 5 points [-]

You've got a more sophisticated notion of causality, but otherwise this is very similar to the causal theory of reference. For example, the way they would describe what's going on with the shadow sneeze variables is that when you name "SNEEZEVAR", there was no causal link that allowed you to pick out the actual SNEEZEVAR: there would need to be a causal arrow going the other way for that to be possible. (and then any later uses of "SNEEZE_VAR" would have to be linked causally to your earlier naming: if we wiped your brain and rebooted it with random noise that happened to be the same, then you wouldn't succeed in referring) I'm pretty sure I've seen someone use a similar kind of example where you can't decide which of two possible things you're referring to because of the lack of a causal link of the right kind.

They also use pretty similar examples: a classic one is to think of an being on the other side of the galaxy thinking about Winston Churchill. Even if they have the right image, they even happen to think the right things about where he lived, what he did etc. it seems that they don't actually succeed in referring to him because of the lack of a causal link. It's just a coincidence.

With that in mind, there are probably arguments made against the causal theory of reference that may apply to you to, but I don't know any off the top of my head.

Comment author: Eliezer_Yudkowsky 21 October 2012 09:24:38PM 3 points [-]

They also use pretty similar examples: a classic one is to think of an being on the other side of the galaxy thinking about Winston Churchill. Even if they have the right image, they even happen to think the right things about where he lived, what he did etc. it seems that they don't actually succeed in referring to him because of the lack of a causal link.

(They could be referring to all objects of class CHURCHILL, but not to our own, particular Churchill, although he happens to be such an object.)

Comment author: Eliezer_Yudkowsky 20 October 2012 08:45:56AM 4 points [-]

Meditation:

If we can only meaningfully talk about parts of the universe that can be pinned down inside the causal graph, where do we find the fact that 2 + 2 = 4? Or did I just make a meaningless noise, there? Or if you claim that "2 + 2 = 4" isn't meaningful or true, then what alternate property does the sentence "2 + 2 = 4" have which makes it so much more useful than the sentence "2 + 2 = 3"?

Comment author: [deleted] 21 October 2012 04:00:24AM 17 points [-]

PA proves "2 + 2 = 4" using the associative property. PA does not prove "2 + 2 = 3". "2 + 2 = 4" is actually shorthand for "((1+1) + (1+1)) = (((1+1)+1)+1)". Moving stuff next to other stuff in our universe happens to follow the associative property; this is why the belief is useful.

Comment author: kpreid 21 October 2012 04:32:04PM *  8 points [-]

I have myself usually seen Peano arithmetic described with 0 and the successor operation (such as in the context of actually implementing it in a computer). in this case,

  S(S(0)) + S(S(0))
= S(S(S(0))) + S(0)
= S(S(S(S(0)))) + 0
= S(S(S(S(0))))

where the two theorems needed are that x + S(y) = S(x) + y and that x + 0 = x. I find this to have less incidental complexity (given that we are interested in working up from axioms, not down from conventional arithmetic) perhaps because the tree of the final expression has no branches. The first theorem can be looked at as expressing that “moving stuff results in the same stuff”, i.e. a conservation law; note that the expression has precisely the same number of nodes.

Comment author: Eliezer_Yudkowsky 21 October 2012 06:14:34AM 5 points [-]

(I like that! The idea that it follows just from the associative property and no other features of PA is quite elegant.)

Comment author: ArisKatsaris 28 October 2012 12:13:11AM *  0 points [-]

This post feel surprisingly insight-bringing to me because the "associative property" can in a sense be considered the "insignificance of parentheses"... and hence both the insignificance of groupings, and the lack of the need to define a starting point of calculation... and in turn these concepts feel connected to the concepts of reductionism and relativity...

Comment author: chaosmosis 27 October 2012 03:54:23AM *  0 points [-]

I would go further and say that without the associative property the concept of numbers, math, and "2 + 2 = 4" does not make sense.

Comment author: MixedNuts 27 October 2012 01:28:12PM 4 points [-]

Octonion multiplication is not associative. Exponentation isn't either; (2^2)^3 = 4^3 = 64, 2^(2^3) = 2^8 = 256. There's likely some kind of useful math with numberlike objects where no interesting operation is associative.

Comment author: chaosmosis 27 October 2012 07:14:56PM *  0 points [-]

How can this math refer to the real world without the associative property? If you can't count the initial "2" then you can't multiply it or anything else, right? Plus, how could we arrive at a concept of exponentation that didn't entail the concept of the associative property?

The pure math might just be too much of an inferential leap for me. I need to see how the math would be created from observations in the real world before I can really understand what you are saying.

Comment author: MixedNuts 27 October 2012 07:40:25PM 7 points [-]

Well, addition of positive integers is associative and has an obvious real-world analogue, so the associophobic math isn't a good choice for describing reality. But if you lived in Bejeweled, addition wouldn't make much sense as a concept - sometimes pushing a thing close to another thing yields two things, sometimes zero. The most fundamental operation would be "flip a pair of adjacent things", which is not associative. (It's sort of a transposition, which would give you group theory, which is full of associative operations, but I don't think you can factor in disappearing rows while preserving associativity - it destroys the bijectivity.)

Comment author: chaosmosis 27 October 2012 09:51:02PM 0 points [-]

Cool, great example.

Comment author: chaosmosis 20 October 2012 11:29:54PM *  8 points [-]

2+2=4 isn't a cause. It's a tautological description. Describing things is useful, though.

Comment author: [deleted] 27 October 2012 06:31:26AM 0 points [-]

I agree...PA was invented based on our observations; our observations aren't just magically predicted by some arbitrary set of rules. PA has only existed since 1889; reality had existed long before that.

Comment author: Bundle_Gerbe 21 October 2012 11:57:12PM *  4 points [-]

I think this example brings out how Pearlian causality differs from other causal theories. For instance, in a counterfactual theory of causation, since the negation of a mathematical truth is impossible, we can't meaningfully think of them as causes.

But in the Pearlian causality it seems that mathematical statements can have causal relations, since we can factor our uncertainty about them, just as we can other statements. I think endoself's comment argues this well. I would add that this is a good example of how causation can be subjective. Before 1984, the Taniyama-Shimura-Weil conjecture and Fermat's last theorem existed as conjectures, and some mathematicians presumably knew about both, but as far as I know they had no clue that they were related. Then Frey conjectured and Ribet proved that the TSW conjecture implies FLT. Then mathematician's uncertainty was such that they would have causal graphs with TSW causing FLT. Now we have a proof of TSW (mostly by Wiles) but any residual uncertainty is still correlated. In the future, maybe there will be many independent proofs of each, and whatever uncertainty is left about them will be (nearly) uncorrelated.

I also think there can be causal relations between mathematical statements and statements about the world. For instance, maybe there is some conjecture of fluid dynamics, which if true would cause us to believe a certain type of wave can occur in certain circumstances. We can make inferences both ways, for instance, if we observe the wave we might increase our credence in the conjecture, and if we prove the conjecture, we might believe the wave can be observed somehow. But it seems that the causal graph would have the conjecture causing the wave. Part of the graph would be:

[̶P̶r̶o̶o̶f̶ ̶o̶f̶ ̶c̶o̶n̶j̶e̶c̶t̶u̶r̶e̶ ̶-̶>̶ ̶c̶o̶n̶j̶e̶c̶t̶u̶r̶e̶ ̶-̶>̶ ̶w̶a̶v̶e̶ ̶<̶-̶ ̶(̶f̶l̶u̶i̶d̶ ̶d̶y̶n̶a̶m̶i̶c̶s̶ ̶a̶p̶p̶l̶i̶e̶s̶ ̶t̶o̶ ̶w̶a̶t̶e̶r̶)̶ ̶]̶

[Proof of conjecture <- conjecture -> wave <- (fluid dynamics applies to water) ]

Comment author: endoself 23 October 2012 05:04:53PM 4 points [-]

Then mathematician's uncertainty was such that they would have causal graphs with TSW causing FLT.

Well the direction of the arrow would be unspecified. After all, not FLT implies not TSW is equivalent to TSW implies FLT, so there's a symmetry here. This often happens in causal modelling; many causal discovery algorithms can output that they know an arrow exists, but they are unable to determine its direction.

Also, conjectures are the causes of their proofs rather than vice versa. You can see this as your degrees of belief in the correctness of purported proofs are independent given that the conjecture is true (or false), but dependent when the truth-value of the conjecture is unknown.

Apart from this detail, I agree with your comment and I find it to be similar to the way I think about the causal structure of math.

Comment author: [deleted] 29 October 2012 08:32:51AM *  2 points [-]

This is very different from how I think about it. Could you expand a little? What do you mean by "when the truth-value of the conjecture is unknown"? That neither C nor ¬C is in your bound agent's store of known theorems?

your degrees of belief in the correctness of purported proofs are independent given that the conjecture is true (or false),

Let S1, S2 be purported single-conclusion proofs of a statement C.

If I know C is false, the purported proofs are trivially independent because they're fully determined incorrect?

Why is S1 independent of S2 given C is true? Are you saying that learning S2⊢C puts C in our theorem bank, and knowing C is true can change our estimation that S1⊢C , but proofs aren't otherwise mutually informative? If so, what is the effect of learning ⊨C on P(S1⊢C)? And why don't you consider proofs which, say, only differ after the first n steps to be dependent, even given the truth of their shared conclusion?

Comment author: endoself 30 October 2012 06:39:47PM 1 point [-]

What do you mean by "when the truth-value of the conjecture is unknown"? That neither C nor ¬C is in your bound agent's store of known theorems?

I meant that the agent is in some state of uncertainty. I'm trying to contrast the case where we are more certain of either C or ¬C with that where we have a significant degree of uncertainty.

If I know C is false, the purported proofs are trivially independent because they're fully determined incorrect?

Yeah, this is just the trivial case.

Why is S1 independent of S2 given C is true?

I was talking about the simple case where there are no other causal links between the two proofs, like common lemmas or empirical observations. Those do change the causal structure by adding extra nodes and arrows, but I was making the simplifying assumption that we don't have those things.

Comment author: Bundle_Gerbe 23 October 2012 09:58:04PM 1 point [-]

Hmm, you are right. Thanks for the correction!

Comment author: Peterdjones 25 October 2012 08:24:52AM -1 points [-]

But in the Pearlian causality it seems that mathematical statements can have causal relations, since we can factor our uncertainty about them, just as we can other statements.

There maybe uncertainty about casual relations and about mathemtical statements, but that does not mean mathematics is causal.

I think endoself's comment argues this well. I would add that this is a good example of how causation can be subjective.

The transition probabilites on a causal diagram may be less than 1, but that represents levels of subjective confidence--epistemology--not causality per se. You can't prove that the universe is indeterministic by writing out a diagram.

Before 1984, the Taniyama-Shimura-Weil conjecture and Fermat's last theorem existed as conjectures, and some mathematicians presumably knew about both, but as far as I know they had no clue that they were related. Then Frey conjectured and Ribet proved that the TSW conjecture implies FLT. Then mathematician's uncertainty was such that they would have causal graphs with TSW causing FLT. Now we have a proof of TSW (mostly by Wiles) but any residual uncertainty is still correlated. In the future, maybe there will be many independent proofs of each, and whatever uncertainty is left about them will be (nearly) uncorrelated.

Yes you can write out a diagram with transitions indicating logical relationships and probabilities representing subjective confidence. But the nodes aren't spatio-temporal events, so it isnt a causal diagram. It is another kind of diagram which happens to have the same structure.

I also think there can be causal relations between mathematical statements and statements about the world.

Causal relations hold between events, not statements.

For instance, maybe there is some conjecture of fluid dynamics, which if true would cause us to believe a certain type of wave can occur in certain circumstances.

What causes us to believe is evidence, not abstract truth.

We can make inferences both ways, for instance, if we observe the wave we might increase our credence in the conjecture, and if we prove the conjecture, we might believe the wave can be observed somehow.

The production of a proof, which is a spatio temproal event, can cause a change in beleif-state, which is a spatio temporal event, which causes changes in behaviour....mathematical truth is not involved. Truth without proof causes nothing. If we dont have reason to believe in a conjecture, we don't act on it, even if it is true.

Comment author: endoself 30 October 2012 06:43:11PM *  2 points [-]

Taboo spatio-temporal. Why is it a good idea to give one category of statements the special name 'events' and to reason about them differently than you would reason about other events?

Also, what about Newcomb's problem?

Comment author: Peterdjones 30 October 2012 06:56:50PM *  -3 points [-]

Newcomb's problem is an irrelevant-to-everything Waste Of Money Brains And Time, AFIAC.

Comment author: Peterdjones 30 October 2012 06:47:49PM *  -1 points [-]

"Events" aren't a kind of statement. However a subset of statements is about events. The point of separating them out is that this discussion is about causality, and, uncontentiously, causality links events. If something is a Non-event (or a statement is not about an event), that is a good argument for not granting it (or what is is about) causal powers.

Comment author: endoself 21 October 2012 02:20:15AM 3 points [-]

It is meaningful to talk about mathematical facts causing other mathematical facts. For example, if I knew the complete laws of physics but did not have enough computing power to determine all their consequences (which would be impossible anyways, as I'm living inside of them), my uncertainty about what is going to happen in the universe would be described by the exact same probability distribution as my uncertainty about the mathematical consequences of the laws of physics, and so both distributions would satisfy the causal Markov condition for the same causal graph* (modulo any uncertainty about whether the laws that I believe to be correct actually do describe the universe).

This works the same way with any other set of mathematical facts. I believe that if the abc conjecture is true, then Szpiro's conjecture is also true and I believe that if the abc conjecture is false, then Shinichi Mochizuki's proof of it is flawed. All of these facts can be put into one probability distribution which can then be factored over a Bayesian network. There is no need to separate the mathematical from the nonmathematical.

* Depending on how exactly you phrase the question, I would even say that these distributions are describing my uncertainty about the same thing, but that isn't necessary here.

Comment author: Eugine_Nier 21 October 2012 03:20:47AM 3 points [-]

The thing is that mathematics seems to have an additional causal structure that seems to be (at least partially) independent from the proof structure.

Comment author: endoself 21 October 2012 01:58:48PM 2 points [-]

I agree with this. I didn't mean to give the impression that the causal structure is the same as the proof structure.

Comment author: Larks 21 October 2012 03:13:52PM 2 points [-]

While it seems viable to say that mathematical truths exert causal (albeit time-invariant) influence on the universe, you might think it prima facie unlikely that we could causally influence them. However, it perhaps we can causally affect the outputs of decision algorithms.

Comment author: The_Duck 21 October 2012 04:55:59AM 2 points [-]

On its own, mathematics is just the study of various systems of rules for symbol manipulation. The "unreasonable effectiveness of mathematics" in more practical areas like physics allows us to conclude something about the structure of physical law. Whatever the true laws of physics are, we conclude, they seem to be well described by certain systems of rules for symbol manipulation.

In our causal models, this is an arrow between "laws of physics" and "observations about the behavior of collections of objects" [subject to Eliezer's caveat about a "laws of physics" node]. Once we include this arrow, we can do neat tricks like counting the numbers of rocks in piles A and B, encoding those numbers in symbols, manipulating those symbols, and finally using the result to correctly predict how many rocks we will have once we combine the two piles.

Comment author: Gust 26 December 2012 03:17:23AM 0 points [-]

I think you've taken EY's question too literally. The real question is about the status of statements and facts of formal systems ("systems of rules for symbol manipulation") in general, not arithmetic, specifically. If you define "mathematics" to include all formal systems, then you can say EY's meditation is about mathematics.

Comment author: somervta 29 October 2012 04:29:28AM 0 points [-]

Unfortunately I accidentally looked at some of the other answers first, but I think my answer would have been this anyway:

The fact "2+2=4" is a fact about the products of certain systems of thought and computation. It is also a abstract description of a certain property of objects in reality (namely, that taking two sets of two objects and combining the sets results in a set of four objects), at least at the classical level.

Comment author: selylindi 22 October 2012 02:34:19PM *  0 points [-]

There seem to be all sorts of reasons that our distant ancestors' development of a number sense was useful enough to be evolutionarily favored. Do we still have everyone in the tribe? Do we outnumber the enemies? How many predators were chasing me? Are these all of my children? A number sense that told us "2+2=3" could be quite maladaptive.

Comment author: CCC 21 October 2012 11:08:22AM 0 points [-]

I see it as being in the causal graph - not as a node, but as an arrow (actually, a whole class of arrows). If I have two stones, and I put two more stones with them, then this will cause me to have four stones. Note that this doesn't apply in all cases - if I have two piles of sand and I put them together with two more piles of sand, the result is one really big pile of sand and not four piles - but it applies in enough cases that the cases in which it does not apply can be considered exceptions for various reasons.

Comment author: endoself 21 October 2012 02:12:01PM 2 points [-]

The problem with this is that something can both be caused and appear in a lot of causal interactions. For example, if I launch a giant mirror into space to block out the sun, all of the arrows from the sun to brightness everywhere have to be changed. In AI, this is often represented using plate notation, where a rectangle (the plate) is drawn around a group of variables that repeat and an arrow from outside the plate effects every instance of the variables in the plate.

Comment author: Eugine_Nier 21 October 2012 10:33:12PM -1 points [-]

Mathematics is a lower, actually the lowest, tier.

Comment author: Gust 26 December 2012 03:11:00AM *  0 points [-]

Actually, if you think of it as affecting us, but not being affected by us, it is, in EY's words, mathematics is higher. We would be "shadows" influenced by the higher tier, but unable to affect it.

But I don't really think this line of reasoning leads anywhere.

Comment author: drnickbone 21 October 2012 10:20:45AM *  10 points [-]

Some thoughts about "epiphenomena" in general, though not related to consciousness.

Suppose there are only finitely many events in the entire history of the universe (or multiverse), so that the universe can be represented by a finite casual graph. If it is an acrylic graph (no causal cycles), then there must be some nodes which are effects but not causes, that is, they are epiphenomena. But then why not posit a smaller graph with the epiphenomenal nodes removed, since they don't do anything? And then that reduced graph is also finite, and also has epiphenomenal nodes.... so why not remove those?

So, is the conclusion that the best model of the universe is a strictly infinite graph, with no epiphenomenal nodes that can be removed e.g. no future big crunches or other singularities? This seems like a dubious piece of armchair cosmology.

Or are there cases where the larger finite graph (with the epiphenomenal nodes) is strictly simpler as a theory than the reduced graph (with the epiphenomena removed), so that Occam's razor tells us to believe in the larger graph? But then Occam's razor is justifying a belief in epiphenomena, which sounds rather odd when put like that!

Comment author: Eliezer_Yudkowsky 21 October 2012 09:18:31PM 7 points [-]

The last nodes are never observed by anyone, but they descend from the same physics, the same F(physics), that have previously been pinned down, or so I assume. You can thus meaningfully talk about them for the same reason you can meaningfully talk about a spaceship going over the cosmological horizon. What we're trying to avoid is SNEEZE_VARs or lower qualia where there's no way that the hypothesis-making agent could ever have observed, inducted, and pinned down the causal mechanism - where there's no way a correspondence between map and territory could possibly be maintained.

Comment author: [deleted] 21 October 2012 10:23:29PM 1 point [-]
Comment author: CCC 21 October 2012 11:26:02AM 1 point [-]

Following this reasoning, if there is a finite causal state machine, then your pruning operation would eventually remove me, you, the human race, the planet Earth.

Now, from inside the universe, I cannot tell whether your hypothesis of a finite state graph universe is true or not - but I do have a certain self-interest in not being removed from existence. I find, therefore, that I am scrambling for justifications for why the finite-state-model universe nodes containing myself are somehow special, that they should not be removed (to be fair, I extend the same justifications to all sentient life).

Comment author: Eugine_Nier 21 October 2012 03:24:47AM 9 points [-]

That said, I can certainly write a computer program in which there's a tier of objects affecting each other, and a second tier - a lower tier - of epiphenomenal objects which are affected by them, but don't affect them.

I would like to point out that any space-like surface (technically 3-fold) divides our universe into two such tiers.

Comment author: Eliezer_Yudkowsky 21 October 2012 09:27:23PM 9 points [-]

Okay, I can see that I need to spell out in more detail one of the ideas here - namely that you're trying to generalize over a repeating type of causal link and that reference is pinned down by such generalization. The Sun repeatedly sends out light in individual Sun-events, electrons repeatedly go on traveling through space instead of vanishing; in a universe like ours, rather than the F(i) being whole new transition tables randomly generated each time, you see the same F(physics) over and over. This is what you can pin down and refer to. Any causal graph is acyclic and can be divided as you say; the surprising thing is that there are no F-types, no causal-link-types, which (over repeating time) descend from one kind of variable to another, without (over time) there being arrows also going back from that kind to the other. Yes, we're generalizing and inducting over time, otherwise it would make no sense to speak of thingies that "affect each other". No two individual events ever affect each other!

Comment author: [deleted] 21 October 2012 10:27:46PM 3 points [-]

Maybe you should elaborate on this in a top-level post.

Comment author: Eliezer_Yudkowsky 25 October 2012 04:29:15AM 2 points [-]

I edited the main post to put it in.

Comment author: William_Quixote 29 October 2012 09:55:31PM 1 point [-]

I will probably reskim the post, but in general it’s not clear to me that editing content into a preexisting posts is better than incorporating then into the next post where it would be appropriate. The former provides the content to all people yet to read it and all people who will reread it while the latter provides the content to all people who yet to read it. So you are trading the time value of getting updated content to the people who will reread this post faster at the expense of not getting updated content to those who will read the next post but not reread the present post.

I don’t have readership and rereadership stats, but this seems like an answerable question.

Comment author: Eugine_Nier 21 October 2012 10:38:23PM 0 points [-]

Okay, I can see that I need to spell out in more detail one of the ideas here - namely that you're trying to generalize over a repeating type of causal link and that reference is pinned down by such generalization.

So in the end, we're back a frequentism.

Also, what about unique events?

Comment author: Cyan 22 October 2012 02:47:55AM *  3 points [-]

Somewhat tangentially, I'd like to point out that simply bringing up relative frequencies of different types of events in discussion doesn't make one a crypto-frequentist -- the Bayesian approach doesn't bar relative frequencies from consideration. In contrast, frequentism does deprecate the use of mathematical probability as a model or representation of degrees of belief/plausibility.

Comment author: Eliezer_Yudkowsky 22 October 2012 12:52:47AM 2 points [-]

Er, no, they're called Dynamic Bayes Nets. And there are no known unique events relative to the fundamental laws of physics; those would be termed "miracles". Physics repeats perfectly - there's no question of frequentism because there's no probabilities - and the higher-level complex events are one-time if you try to measure them precisely; Socrates died only once, etc.

Comment author: Eugine_Nier 23 October 2012 12:44:37AM 0 points [-]

And there are no known unique events relative to the fundamental laws of physics

What about some of the things going on at the LHC?

Comment author: CCC 21 October 2012 10:58:25AM 3 points [-]

So... the past can affect the future, but the future cannot affect the past (barring things like tachyons, which may or may not exist in any case). Taking this in context of the article could lead to an interesting discussion on the nature of time; does the past still exist once it is past? Does the future exist yet in any meaningful way?

Comment author: [deleted] 21 October 2012 01:07:24PM 3 points [-]

I'd say "No, unless you're using the words still and yet in a weird way."

Comment author: CCC 21 October 2012 07:32:24PM 2 points [-]

Consider for a moment the concept of the lack of simultaneity in special relativity.

Consider, specifically, the train and platform experiment. A train passes a platform travelling at a significant fraction of the speed of light; some time before it does so, a light bulb in the precise centre of the train flashes, once.

The observer T on the train will find that the light reaches the front and back of the train simultaneously; the observer P on the platform finds that the light hits the back of the train before it hits the front of the train.

Consider now the instant in which the train observer T says that the light is currently hitting the front and the back of the train simultaneously. At that precise instant, he glances out of the window and sees that he is right next to the platform observer P. the event "P and T pass each other" occurs at that same instant. Thus, all three events - the light hitting the front of the train, the light hitting the rear of the train, and P and T passing, are simultaneous.

Now consider P. In P's reference frame, these events are not simultaneous. The occur in the following order:

  • The light hits the rear of the train
  • T passes P
  • The light hits the front of the train

So. In the instant in which T and P pass each other, does the event "the light hits the rear of the train" exist? It is not in the past, or future, lightcone of the event "T and P pass each other", and thus cannot be directly causally linked to that event (barring FTL travel, which causes all sorts of headaches for relativity in any case).

Comment author: [deleted] 21 October 2012 07:47:03PM *  2 points [-]

In the instant in which T and P pass each other, does the event "the light hits the rear of the train" exist?

The phrase "In the instant in which T and P pass each other" has a different meaning (namely, it refers to a different spacelike hypersurface) depending on what frame of reference the speaker is using. Some of those hypersurfaces include the event "the light hits the rear of the train" and others don't.

Comment author: CCC 23 October 2012 07:25:25AM 1 point [-]

That is true. Nonetheless, you have two observers, T and P, who disagree at a given moment on whether the event "the light hits the rear of the train" is currently happening, or whether it has already happened. (Another observer can be introduced for whom the event has not yet happened).

So. If the present exists - that is, if everything which is on the same spacelike hypersurface as me at this current moment exists - then every possible spacelike hypersurface including me must exist. Which means that, over in the Andromeda galaxy, there must be quite a large time interval that exists all at once (everything that's not in my past/future light cone). Applying the same argument to a hypothetical observer in the Andromeda Galaxy implies that a large swath of time over here must all be in existance as well.

Now, it is possible that there is one particular spacelike hypersurface that can be considered to be the only spacelike hypersurface in existance at any given time; if this were the case, though, then I would expect that there would be some experiment that could demonstrate which spacelike hypersurface it is. That same experiment would disprove special relativity, and require an updating of that theory. On the other hand, if the past and future are in existance in some way, I would expect that there would be some way, as yet undiscovered, to affect them - some way, in short, to travel through time (or at least to send an SMS to the past). Either way, it leads to interesting possibilities for future physics.

Comment author: [deleted] 23 October 2012 09:22:28AM *  0 points [-]

First of all, natural language sucks at specifying whether a statement is indexical/deictic (its referent depends on who is speaking, and where and when they are speaking, etc.) or not. The compulsory tense marking on verbs is part of the problem (“there is” is usually but not always taken to mean “there is now” as “is” is in the present tense, and that's indexical -- “now” refers to a different time in this comment than in one written three years ago), though not the only one (“it's not raining” is usually taken to mean “it's not raining here”, not “it's not raining anywhere”).

Now, it is possible that there is one particular spacelike hypersurface that can be considered to be the only spacelike hypersurface in existance at any given time

Yes, once you specify (explicitly or implicitly) what you mean by “at any given time” (i.e. what frame of reference you're using). But there's no God-given choice for that. (Well, there's the frame of reference in which the dipole anisotropy of the cosmic microwave background vanishes, but you have to “look out” to know what that is; in a closed system you couldn't tell.) IOW the phrase “at any given time” must also be taken as indexical; in everyday life that doesn't matter much because the hypersurfaces you could plausibly be referring to are only separated by tiny fractions of a second in the regions you'd normally want to talk about.

On the other hand, if the past and future are in existance in some way,

Yes, if “are” is interpreted non-indexically (i.e. not as “are now”).

I would expect that there would be some way, as yet undiscovered, to affect them - some way, in short, to travel through time (or at least to send an SMS to the past).

Why? In special relativity, “X can affect Y” is equivalent to “Y is within or on the future light cone of X”, which is a partial order relation, and that's completely self-consistent. (But in the real world, special relativity only applies locally, and even there we can't be 100% sure it applies exactly and in all conditions.)

Comment author: CCC 23 October 2012 06:12:11PM 1 point [-]

Yes, once you specify (explicitly or implicitly) what you mean by “at any given time” (i.e. what frame of reference you're using).

This is where it all gets complicated. If I'm trying to talk about one instantaneous event maintaining an existence for longer than an instant - well, language just isn't structured right for that. An event can partake of many frames of reference, many of which can include me at different times by my watch (particularly if the event in question takes place in the Andromeda Galaxy). So, if there is one reference frame where an Event occurs at the same time as my watch shows 20:00, and another reference frame shows the same (distant) event happening while my watch says 21:00, then does that Event remain in existence for an entire hour?

That's basically the question I'm asking; while I suspect that the answer is 'no', I also don't see what experiment can be used to prove either a positive or a negative answer to that question (and either way, the same experiment seems likely to also prove something else interesting).

Yes, if “are” is interpreted non-indexically (i.e. not as “are now”).

I meant it as "are now".

I would expect that there would be some way, as yet undiscovered, to affect them - some way, in short, to travel through time (or at least to send an SMS to the past).

Why? In special relativity, “X can affect Y” is equivalent to “Y is within or on the future light cone of X”, which is a partial order relation, and that's completely self-consistent.

Because if it is now in existence, then I imagine that there is now some way to affect it; which in this case would imply time travel (and therefore at least some form of FTL travel)

Comment author: [deleted] 24 October 2012 01:22:40PM 2 points [-]

First of all, Thou Shalt Not use several frames of reference at once unless you know what you're doing or you risk being badly confused. (Take a look at the Special Relativity section of the Usenet Physics FAQ, especially the discussion of the Twin Paradox.) Possibly, get familiar with spacetime diagrams (also explained in that FAQ).

According to special relativity, the duration of the set of instants B in your life such as there exists an inertial frame of reference such that B is simultaneous with a fixed event A happening in Andromeda is 2L/c, where L is the distance from you to Andromeda. (Now you do need experiments to tell whether special relativity applies to the real world, but any deviation from it --except due to gravitation-- must be very small or only apply to certain circumstances, or we would have seen it by now.)

I meant it as "are now".

I'd say that the concept of “now” needs a frame of reference to be specified (or implicit from the context) to make sense.

Because if it is now in existence, then I imagine that there is now some way to affect it; which in this case would imply time travel (and therefore at least some form of FTL travel)

I think you are trying to apply to Minkowski spacetime an intuition that only applies to Galilean spacetime (and even then, it's not an intuition that everyone shares; IIRC, there have been people thinking that instant action at a distance is counterintuitive and a reason to suspect that Newtonian physics is not the whole story for centuries, even before Einstein came along).

Comment author: CCC 25 October 2012 08:23:36AM 0 points [-]

First of all, Thou Shalt Not use several frames of reference at once unless you know what you're doing or you risk being badly confused.

I think that this is important; I have come to suspect that I am somewhat confused.

I think you are trying to apply to Minkowski spacetime an intuition that only applies to Galilean spacetime

This is more than likely correct. I would also note that I have been applying, over very long (intergalactic) distances, the assumption that there is no expansion, which is clearly wrong. I suspect that I should probably look more into General Relativity before continuing along this train of thought.

Comment author: RichardChappell 22 October 2012 10:43:22PM *  7 points [-]

FWIW, my old post 'Zombie Rationality' explores what I think the epiphenomenalist should say about the worry that "the upper-tier brain must be thinking meaningless gibberish when the upper-tier lips [talk about consciousness]"

One point to flag is that from an epiphenomenalist's perspective, mere brains never really mean anything, any more than squiggles of ink do; any meaning we attribute to them is purely derivative from the meaning of appropriately-related thoughts (which, on this view, essentially involve qualia).

Another thing to flag is that epiphenomenalism needn't imply that our thoughts are causally irrelevant, but merely their experiential component. It'd be a mistake to identify oneself with just one's qualia (as Eliezer seems to attribute to the epiphenomenalist). It's true that our qualia don't write philosophy papers about consciousness. But we, embodied conscious persons, do write such papers. Of course, the causal explanation of the squiggles depends only on our physical parts. But the fact that the squiggles are about consciousness (or indeed anything at all) depends crucially upon the epiphenomenal aspects of our minds, in addition.

Comment author: novalis 25 October 2012 08:42:15PM 1 point [-]

Where in the causal diagram does "meaning" go?

Comment author: RichardChappell 25 October 2012 11:21:37PM 2 points [-]

Meaning doesn't seem to be a thing in the way that atoms and qualia are, so I'm doubtful that the causal criterion properly applies to it (similarly for normative properties).

(Note that it would seem rather self-defeating to claim that 'meaning' is meaningless.)

Comment author: Benito 26 October 2012 07:18:35PM 1 point [-]

What exactly do you mean by "mean"?

Comment author: RichardChappell 27 October 2012 06:17:44PM 0 points [-]

I couldn't help one who lacked the concept. But assuming that you possess the concept, and just need some help in situating it in relation to your other concepts, perhaps the following might help...

Our thoughts (and, derivatively, our assertions) have subject-matters. They are about things. We might make claims about these things, e.g. claiming that certain properties go together (or not). When I write, "Grass is green", I mean that grass is green. I conjure in my mind's eye a mental image of blades of grass, and their colour, in the image, is green. So, I think to myself, the world is like that.

Could a zombie do all this? They would go "through the motions", so to speak, but they wouldn't actually see any mental image of green grass in their mind's eye, so they could not really intend that their words convey that the world is "like that". Insofar as there are no "lights on inside", it would seem that they don't really intend anything; they do not have minds.

If you can understand the above two paragraphs, then it seems that you have a conception of meaning as a distinctively mental relation (e.g. that holds between thoughts and worldly objects or states of affairs), not reducible to any of the purely physical/functional states that are shared by our zombie twins.

Comment author: Eliezer_Yudkowsky 28 October 2012 05:53:26AM 2 points [-]

(From "The Simple Truth", a parable about using pebbles in a bucket to keep count of the sheep in a pasture.)

“My pebbles represent the sheep!” Autrey says triumphantly. “Your pebbles don’t have the representativeness property, so they won’t work. They are empty of meaning. Just look at them. There’s no aura of semantic content; they are merely pebbles. You need a bucket with special causal powers.”

“Ah!” Mark says. “Special causal powers, instead of magic.”

“Exactly,” says Autrey. “I’m not superstitious. Postulating magic, in this day and age, would be unacceptable to the international shepherding community. We have found that postulating magic simply doesn’t work as an explanation for shepherding phenomena. So when I see something I don’t understand, and I want to explain it using a model with no internal detail that makes no predictions even in retrospect, I postulate special causal powers. If that doesn’t work, I’ll move on to calling it an emergent phenomenon.”

“What kind of special powers does the bucket have?” asks Mark.

“Hm,” says Autrey. “Maybe this bucket is imbued with an about-ness relation to the pastures. That would explain why it worked – when the bucket is empty, it means the pastures are empty.”

“Where did you find this bucket?” says Mark. “And how did you realize it had an about-ness relation to the pastures?”

“It’s an ordinary bucket,” I say. “I used to climb trees with it… I don’t think this question needs to be difficult.”

“I’m talking to Autrey,” says Mark.

“You have to bind the bucket to the pastures, and the pebbles to the sheep, using a magical ritual – pardon me, an emergent process with special causal powers – that my master discovered,” Autrey explains.

Autrey then attempts to describe the ritual, with Mark nodding along in sage comprehension.

“And this ritual,” says Mark, “it binds the pebbles to the sheep by the magical laws of Sympathy and Contagion, like a voodoo doll.”

Autrey winces and looks around. “Please! Don’t call it Sympathy and Contagion. We shepherds are an anti-superstitious folk. Use the word ‘intentionality’, or something like that.”

“Can I look at a pebble?” says Mark.

“Sure,” I say. I take one of the pebbles out of the bucket, and toss it to Mark. Then I reach to the ground, pick up another pebble, and drop it into the bucket.

Autrey looks at me, puzzled. “Didn’t you just mess it up?”

I shrug. “I don’t think so. We’ll know I messed it up if there’s a dead sheep next morning, or if we search for a few hours and don’t find any sheep.”

“But -” Autrey says.

“I taught you everything you know, but I haven’t taught you everything I know,” I say.

Mark is examining the pebble, staring at it intently. He holds his hand over the pebble and mutters a few words, then shakes his head. “I don’t sense any magical power,” he says. “Pardon me. I don’t sense any intentionality.”

“A pebble only has intentionality if it’s inside a ma- an emergent bucket,” says Autrey. “Otherwise it’s just a mere pebble.”

“Not a problem,” I say. I take a pebble out of the bucket, and toss it away. Then I walk over to where Mark stands, tap his hand holding a pebble, and say: “I declare this hand to be part of the magic bucket!” Then I resume my post at the gates.

Autrey laughs. “Now you’re just being gratuitously evil.”

I nod, for this is indeed the case.

“Is that really going to work, though?” says Autrey.

I nod again, hoping that I’m right. I’ve done this before with two buckets, and in principle, there should be no difference between Mark’s hand and a bucket. Even if Mark’s hand is imbued with the elan vital that distinguishes live matter from dead matter, the trick should work as well as if Mark were a marble statue.

(The moral: In this sequence, I explained how words come to 'mean' things in a lawful, causal, mathematical universe with no mystical subterritory. If you think meaning has a special power and special nature beyond that, then (a) it seems to me that there is nothing left to explain and hence no motivation for the theory, and (b) I should like you to say what this extra nature is, exactly, and how you know about it - your lips moving in this, our causal and lawful universe, the while.)

Comment author: Psy-Kosh 28 October 2012 07:03:49AM *  2 points [-]

How, precisely, does one formalize the concept of "the bucket of pebbles represents the number of sheep, but it is doing so inaccurately." ie, that it's a model of the number of sheep rather than about something else, but a bad/inaccurate model?

I've fiddled around a bit with that, and I find myself passing a recursive buck when I try to precisely reduce that one.

The best I can come up with is something like "I have correct models in my head for the bucket, pebbles, sheep, etc, individually except that I also have some causal paths linking them that don't match the links that exist in reality."

Comment author: fubarobfusco 28 October 2012 07:28:47AM *  1 point [-]

See this thread for a discussion. A less buck-passing model is: "This bucket represents the sheep ... plus an error term resulting from this here specific error process."

For instance, if I systematically count two sheep exiting together as one sheep, then the bucket represents the number of sheep minus the number of sheep-pairs erroneously detected as one sheep. It's not enough to say the sheep-detector is buggy; to have an accurate model of what it does (and thus, what its representations mean) you need to know what the bug is.

Comment author: RichardChappell 28 October 2012 03:56:30PM 3 points [-]

It's a nice parable and all, but it doesn't seem particularly responsive to my concerns. I agree that we can use any old external items as tokens to model other things, and that there doesn't have to be anything "special" about the items we make use of in this way, except that we intend to so use them. Such "derivative intentionality" is not particularly difficult to explain (nor is the weak form of "natural intentionality" in which smoke "means" fire, tree rings "signify" age, etc.). The big question is whether you can account for the fully-fledged "original intentionality" of (e.g.) our thoughts and intentions.

In particular, I don't see anything in the above excerpt that addresses intuitive doubts about whether zombies would really have meaningful thoughts in the sense familiar to us from introspection.

Comment author: Eliezer_Yudkowsky 29 October 2012 12:36:23AM 2 points [-]

“I toss in a pebble whenever a sheep passes,” I point out.

“When a sheep passes, you toss in a pebble?” Mark says. “What does that have to do with anything?”

“It’s an interaction between the sheep and the pebbles,” I reply.

“No, it’s an interaction between the pebbles and you,” Mark says. “The magic doesn’t come from the sheep, it comes from you. Mere sheep are obviously nonmagical. The magic has to come from somewhere, on the way to the bucket.”

I point at a wooden mechanism perched on the gate. “Do you see that flap of cloth hanging down from that wooden contraption? We’re still fiddling with that – it doesn’t work reliably – but when sheep pass through, they disturb the cloth. When the cloth moves aside, a pebble drops out of a reservoir and falls into the bucket. That way, Autrey and I won’t have to toss in the pebbles ourselves.”

Mark furrows his brow. “I don’t quite follow you… is the cloth magical?”

I shrug. “I ordered it online from a company called Natural Selections. The fabric is called Sensory Modality.” I pause, seeing the incredulous expressions of Mark and Autrey. “I admit the names are a bit New Agey. The point is that a passing sheep triggers a chain of cause and effect that ends with a pebble in the bucket."

Comment author: RichardChappell 29 October 2012 03:50:59AM 5 points [-]

And this responds to what I said... how?

Comment author: Eliezer_Yudkowsky 29 October 2012 03:57:32AM 7 points [-]

I can build an agent that tracks how many sheep are in the pasture using an internal mental bucket, and keeps looking for sheep until they're all returned. From an outside standpoint, this agent's mental bucket is meaningful because there's a causal process that correlates it to the sheep, and this correlation is made use of to steer the world into futures where all sheep are retrieved. And then the mysterious sensation of about-ness is just what it feels like from the inside to be that agent, with a side order of explicitly modeling both yourself and the world so that you can imagine that your map corresponds to the territory, with a side-side order of your brain making the simplifying assumption that (your map of) the map has a primitive intrinsic correspondence to (your map of) the territory.

In actuality this correspondence is not the primitive and local quality it feels like; it's maintained by the meeting of hypotheses and reality in sense data. A third party or reflecting agent would be able to see the globally maintained correspondence by simultaneously tracing back actual causes of sense data and hypothesized causes of sense data, but this is a chain property involving real lattices of causal links and hypothetical lattices of causal links meeting in sense data, not an intrinsic quality of a single node in the lattice considered in isolation from the senses and the hypotheses linking it to the senses.

So far as I can tell, there's nothing left to explain.

--

“At exactly which point in the process does the pebble become magic?” says Mark.

“It… um…” Now I’m starting to get confused. I shake my head to clear away cobwebs. This all seemed simple enough when I woke up this morning, and the pebble-and-bucket system hasn’t gotten any more complicated since then. “This is a lot easier to understand if you remember that the point of the system is to keep track of sheep.”

Comment author: novalis 26 October 2012 12:10:12AM *  1 point [-]

I'm trying to figure out what work "meaning" is doing. Eliezer says brains are "thinking" meaningless gibberish. You dispute this by saying,

... mere brains never really mean anything, any more than squiggles of ink do; any meaning we attribute to them is purely derivative from the meaning of appropriately-related thoughts ...

But what are brains thinking, if not thoughts?

And then

But the fact that the squiggles are about consciousness (or indeed anything at all) depends crucially upon the epiphenomenal aspects of our minds, in addition.

This implies that "about"-ness and "meaning" have roughly the same set of properties. But I don't understand why anyone believes anything about "meaning" (in this sense). If it doesn't appear in the causal diagram, how could we tell that we're not living in a totally meaningless universe? Let's play the Monday-Tuesday game: on Monday, our thoughts are meaningful; on Tuesday, they're not. What's different?

Comment author: RichardChappell 26 October 2012 12:36:34AM *  1 point [-]

But what are brains thinking, if not thoughts?

Right, according to epiphenomenalists, brains aren't thinking (they may be computing, but syntax is not semantics).

If it doesn't appear in the causal diagram, how could we tell that we're not living in a totally meaningless universe?

Our thoughts are (like qualia) what we are most directly acquainted with. If we didn't have them, there would be no "we" to "tell" anything. We only need causal connections to put us in contact with the world beyond our minds.

Comment author: novalis 26 October 2012 12:43:18AM 3 points [-]

So if we taboo "thinking" and "computing", what is it that brains are not doing?

Comment author: RichardChappell 26 October 2012 01:12:42AM 1 point [-]

You can probably give a functionalist analysis of computation. I doubt we can reductively analyse "thinking" (at least if you taboo away all related mentalistic terms), so this strikes me as a bedrock case (again, like "qualia") where tabooing away the term (and its cognates) simply leaves you unable to talk about the phenomenon in question.

Comment author: novalis 26 October 2012 03:31:47PM 3 points [-]

It sounds like "thinking" and "qualia" are getting the special privilege of being irreducible, even though there have been plenty of attempts to reduce them, and these attempts have had at least some success. Why can't I pick any concept and declare it a bedrock case? Is my cat fuzzy? Well, you could talk about how she is covered with soft fur, but it's possible to imagine something fuzzy and not covered with fur, or something covered with fur but not fuzzy. Because it's possible to imagine these things, clearly fuzziness must be non-physical. It's maybe harder to imagine a non-fuzzy cat than to imagine a non-thinking person, but that's just because fuzziness doesn't have the same aura of the mysterious that thinking and experiencing do.

Comment author: RichardChappell 27 October 2012 06:30:19PM 0 points [-]

Well, you could talk about how she is covered with soft fur, but it's possible to imagine something fuzzy and not covered with fur, or something covered with fur but not fuzzy. Because it's possible to imagine these things, clearly fuzziness must be non-physical.

Erm, this is just poor reasoning. The conclusion that follows from your premises is that the properties of fuzziness and being-covered-in-fur are distinct, but that doesn't yet make fuzziness non-physical, since there are obviously other physical properties besides being-covered-in-fur that it might reduce to. The simple proof: you can't hold ALL the other physical facts fixed and yet change the fuzziness facts. Any world physically identical to ours is a world in which your cat is still fuzzy. (There are no fuzz-zombies.) This is an obvious conceptual truth.

So, in short, the reason why you can't just "pick any concept and declare it a bedrock case" is that competent conceptual analysis would soon expose it to be a mistake.

Comment author: novalis 28 October 2012 03:22:38AM 2 points [-]

No, I'm saying that you could hold all of the physical facts fixed and my cat might still not be fuzzy. This is somewhat absurd, but I have a tremendously good imagination; if I can imagine zombies, I can imagine fuzz-zombies.

Comment author: Peterdjones 26 October 2012 04:19:00PM 0 points [-]

I don't beleive anyone has regarded thinking as causally irreducible for at least a century. Could you cite a partially successful reduction of qualia?

Comment author: pragmatist 26 October 2012 04:32:53PM 3 points [-]

I don't beleive anyone has regarded thinking as causally irreducible for at least a century.

Read the parent of the comment you're responding to.

Comment author: [deleted] 26 October 2012 04:54:26PM *  0 points [-]

It depends on what you mean by 'thinking', but I think the view is pretty widespread that rational relations (like the relation of justification between premises and a conclusion) are not reducible to any physical relation in such a way that explains or even preserves the rational relation.

I'm thinking of Donald Davidson's 'Mental Events' as an example at the moment, just to illustrate the point. He would say that while every token mental state is identical to a token physical state, and every token mental causal relation (like a relation of inference or justification) is identical to a token physical causal relation...

...nevertheless, types of mental states, like the thought that it is raining, and types of mental causal relations, like the inference that if it is raining, and I don't want to get wet, then I should bring an umbrella, are not identical to types of physical states or types of physical causal relations.

This has the result that 1) we can be assured that the mind supervenes on the brain in some way, and that there's nothing immaterial or non-physical going on, but 2) there are in principle no explanations of brain states and relations which suffice as explanations of anything like thinking, reasoning, inferring etc..

Comment author: novalis 26 October 2012 04:22:22PM 0 points [-]

Dennett: Consciousness Explained.

Comment author: Peterdjones 26 October 2012 04:44:29PM 0 points [-]

I doubt we can reductively analyse "thinking"

If we cant even get a start on that, how did we get a start on building AI?

Comment author: RichardChappell 27 October 2012 06:20:51PM 1 point [-]

I'm not sure I follow you. Why would you need to analyse "thinking" in order to "get a start on building AI"? Presumably it's enough to systematize the various computational algorithms that lead to the behavioural/functional outputs associated with intelligent thought. Whether it's really thought, or mere computation, that occurs inside the black box is presumably not any concern of computer scientists!

Comment author: Peterdjones 30 October 2012 02:41:27AM 0 points [-]

I'm not sure I follow you. Why would you need to analyse "thinking" in order to "get a start on building AI"?

Becuase thought is essential to intelligence. Why would you need to analyse intelligence to get a start on building artificial intelliigence? Because you would have no idea what you were tryinng to do if you didn't.

Presumably it's enough to systematize the various computational algorithms that lead to the behavioural/functional outputs associated with intelligent thought.

I fail to see how that is not just a long winded way of saying "analysing thought"

Comment author: ialdabaoth 25 October 2012 10:24:17PM 0 points [-]

Before we answer that question, we need to pick a meaning for "meaning" that means something.

(parse that a few times with your Gödel hat on)

Comment author: Eugine_Nier 26 October 2012 01:49:14AM 1 point [-]

You use the word "mean" twice in that sentence outside quotes, so whatever you mean by it.

Comment author: bryjnar 21 October 2012 10:56:10AM 3 points [-]

It's not clear to me why you're allowing the possibility of an upper-tier universe. In particular, a similar kind of example to one you give suggests that it's not meaningful to talk about being in a sufficiently good simulation.

Suppose we're wondering whether we're in a simulation or not. But there are a couple of possibilities. In one of them, the alien simulation-runner's mug of coffee is green, and in the other it's blue. But both of these situations have (to abuse notation somewhat) the same causal arrows pointing into our experience, and so we can't meaningfully talk about them (if I understand your argument).

That reminds me a lot of Putnam's argument that we can't be "brains in vats", which also proceeds from similar thoughts about reference. (The above is a butchered translation of it!) I'd recommend reading it if you haven't - it's not the clearest thing ever, but he's riffing on the same themes. But I think the conclusion tells against the argument: we surely can talk meaningfully about things like being "brains in a vat".

Comment author: mfb 25 October 2012 03:12:00PM 6 points [-]

I think the question "does consciousness affect neurons?" is as meaningful as "does the process of computation in a computer affect bits?".

Comment author: Eliezer_Yudkowsky 26 October 2012 05:57:35AM 4 points [-]

In other words, "Yes"?

Comment author: mfb 26 October 2012 04:06:21PM *  1 point [-]

Bit modifications are part of the process of computation. I wouldn't say they are "affected by" that (they depend causally on the input which started the process of computation, however). In a similar way, individual humans are not affected by the concept of "mankind" for all of them.

Comment author: dspeyer 21 October 2012 01:59:53AM 6 points [-]

Can anyone explain why epiphenomenalist theories of consciousness are interesting? There have been an awful lot of words on them here, but I can't find a reason to care.

Comment author: drethelin 21 October 2012 02:20:18AM 12 points [-]

Pretty much the same reason religion needs to be talked about. If no one had invented it wouldn't be useful to dispute notions of god creating us for a divine purpose, but because many people think this indeed happened you have to talk about it. It's especially important for reasonable discussions of AI.

Comment author: dspeyer 21 October 2012 06:47:27AM 4 points [-]

Religion and epiphenomenalogy differ in three important ways:

  • Religion is widespread. Almost everyone knows what it is. Most people have at least some religious memes sticking in their heads. A significant fraction of people have dangerous religious memes in their heads so decreasing those qualifies as raising the sanity waterline. Epiphenomenalogy is essentially unknown outside academic philosophy, and now the lesswrong readership.
  • Religion has impact everywhere. People have died because of other people's religious beliefs, and not just from violence. Belief in epiphenomenalogy has almost no impact on the lives of non-believers.
  • Religious thought patterns re-occur. Authority, green/blue, and "it is good to believe" show up over and over again. The sort of thoughts that lead to epiphenomenalogy are quite obscure.
Comment author: Kaj_Sotala 21 October 2012 08:47:39AM 5 points [-]

Epiphenomenalogy is essentially unknown outside academic philosophy, and now the lesswrong readership.

I'd say it's more widespread than that. Some strands of Buddhist thought, for instance, seem to strongly imply it even if they didn't state it outright. And it feels like it'd be the most intuitive way of thinking about consciousness for many of the people who'd think about it at all, even if they weren't familiar with academic philosophy. (I don't think I got it from academic philosophy, though I can't be sure of that.)

Comment author: hairyfigment 21 October 2012 06:31:10PM 3 points [-]

The word "epiphenomenalogy" is rare. The actual theory seems like an academic remnant of the default belief that 'You can't just reduce everything to numbers, people are more than that.'

So your last point seems entirely wrong. Zombie World comes from the urge to justify religious dualism or say that it wasn't all wrong (not in essence). And the fact that someone had to take it this far shows how untenable dualism seems in a practical sense, to educated people.

Comment author: CarlShulman 21 October 2012 08:25:33PM 11 points [-]

It seems that you get similar questions as a natural outgrowth of simple computational models of thought. E.g. if one performs Solomonoff induction on the stream of camera inputs to a robot, what kind of short programs will dominate the probability distribution over the next input? Not just programs that simulate the physics of our universe: one would also need additional code to "read off" the part of the simulated universe that corresponded to the camera inputs. That additional code looks like epiphenomenal mind-stuff. Using this framework you can pose questions like "if the camera is expected to be rebuilt using different but functionally equivalent materials, will his change the inputs Solomonoff induction predicts?" or "if the camera is about to be duplicated, which copy's inputs will be predicted by Solomonoff induction?"

If we go beyond Solomonoff induction to allow actions, then you get questions that map pretty well to debates about "free will."

Comment author: Giles 28 October 2012 08:59:33PM 1 point [-]

Oh wow... I had been planning on writing a discussion post on essentially this topic. One quick question - if you have figured out the shortest program that will generate the camera data, is there a non-arbitrary way we can decide which parts of the program correspond to "physics of our universe" and which parts correspond to "reading off camera's data stream within universe"?

Comment author: Matt_Simpson 28 October 2012 09:48:43PM 0 points [-]

one would also need additional code to "read off" the part of the simulated universe that corresponded to the camera inputs. That additional code looks like epiphenomenal mind-stuff.

I don't understand why the additional code looks like epiphenomenal mind-stuff. Care to explain?

Comment author: gwern 28 October 2012 11:12:00PM 2 points [-]

I take Carl to meaning that: the program corresponding to 'universe A simulating universe B and I am in universe B' is strictly more complex than 'I am in universe B' while also predicting all the same observations, and so the 'universe A simulating universe B' part of the program makes no difference in the same way that mental epiphenomena make no difference - they predict you will make the same observations, while being strictly more complex.

Comment author: CarlShulman 29 October 2012 09:18:44PM 0 points [-]

This seems to be talking about something entirely different.

Comment author: SilasBarta 28 October 2012 11:59:15PM 0 points [-]

the program corresponding to 'universe A simulating universe B and I am in universe B' is strictly more complex than 'I am in universe B' while also predicting all the same observations, and so the 'universe A simulating universe B' part of the program makes no difference in the same way that mental epiphenomena make no difference - they predict you will make the same observations, while being strictly more complex.

True, but, just as a reminder, that's not the position we're in. There are other (plausibly necessary) parts of our world model that could give us the implication "universe A simulates us" "for free", just as we get "the electron that goes beyond our cosmological horizon keeps existing" is an implication we get "for free" as a result of minimal models of physics.

In this case (per the standard Simulation Argument), the need to resolve the question of "what happens in civilizations that can construct virtual worlds indistinguishable from non-virtual worlds" can force us to posit parts of a (minimal) model that then imply the existence of universe A.

Comment author: Matt_Simpson 28 October 2012 11:32:38PM 0 points [-]

Ah, ok, that makes sense. Thanks!

Comment author: CarlShulman 29 October 2012 09:17:30PM 1 point [-]

The code simulating a physical universe doesn't need to make any reference to which brain or camera in the simulation is being "read off" to provide the sensory input stream. The additional code takes the simulation, which is a complete picture of the world according to the laws of physics as they are seen by the creatures in the simulation, and outputs a sensory stream. This function is directly analogous to what dualist/epiphenomenalist philosopher of mind David Chalmers calls "psychophysical laws."

Comment author: Gust 26 December 2012 04:15:24AM 0 points [-]

I don't know if this insight is originally yours or not, but thank you for it. It's like you just gave me a piece of the puzzle I was missing (even if I still don't know where it fits).

Comment author: AnotherIdiot 21 October 2012 02:16:45AM *  6 points [-]

Because epiphenomenalist theories are common but incorrect, and the goal of LessWrong is at least partially what its name implies.

Comment author: Eliezer_Yudkowsky 21 October 2012 06:13:02AM 6 points [-]

It's good practice for seeing how the rules play out; and having a clear mental visualization of how causality works, and what sort of theories are allowed to be meaningful, is actually reasonably important both for anti-nonsense techniques and AI construction. People who will never need any more skill than they currently have in Resist Nonsense or Create Friendly AI can ignore it, I suppose.

Comment author: bryjnar 21 October 2012 10:30:50AM 1 point [-]

Philosophers have a tendency to name pretty much every position that you can hold by accepting/refusing various "key" propositions. Epiphenomenalism tends to be reached by people frantically trying to hold on to their treasured beliefs about the way the mind works. Then they realise they can consistently be epiphenomenalists and they feel okay because it has a name or something.

Basically, it's a consistent position (well, Eliezer seems to think it's meaningless!), and so you want to go to some effort to show that it's actually wrong. Plus it's a good exercise to think about why it's wrong.

Comment author: RichardChappell 25 October 2012 11:10:27PM 1 point [-]

In my experience, most philosophers are actually pretty motivated to avoid the stigma of "epiphenomenalism", and try instead to lay claim to some more obscure-but-naturalist-friendly label for their view (like "non-reductive physicalism", "anomalous monism", etc.)

Comment author: Kaj_Sotala 21 October 2012 08:01:37AM *  5 points [-]

EDIT: After thinking things through, I concluded that Eliezer was right, and that epiphenomalism was indeed confused and incoherent. Leaving this comment here as a record of how I came to agree with that conclusion.

The closest theory to this which definitely does seem coherent - i.e., it's imaginable that it has a pinpointed meaning - would be if there was another little brain living inside my brain, made of shadow particles which could affect each other and be affected by my brain, but not affect my brain in turn. This brain would correctly hypothesize the reasons for its sensory experiences - that there was, from its perspective, an upper tier of particles interacting with each other that it couldn't affect. Upper-tier particles are observable, i.e., can affect lower-tier senses, so it would be possible to correctly induct a simplest explanation for them. And this inner brain would think, "I can imagine a Zombie Universe in which I am missing, but all the upper-tier particles go on interacting with each other as before." If we imagine that the upper-tier brain is just a robotic sort of agent, or a kitten, then the inner brain might justifiably imagine that the Zombie Universe would contain nobody to listen - no lower-tier brains to watch and be aware of events.

Positing "another little brain" to act as the epiphenomenal component sounds unnecessarily complicated to me. You mentioned earlier the possibility of programming a simulation with balls that bounce off each other, and then some little shadows that followed the balls around. It is not obvious to me why "balls that bounce off each other" couldn't be the physical activity of the neurons, "little shadows that followed the balls around" couldn't be the qualia that were produced as a side-effect of the neurons' physical activity.

I think - though I might have misunderstood - that you are trying to exclude this possibility with the "How could you possibly know about the lower tier, even if it existed?" question, and the suggestion that we could only know about the shadows because we can look at the simulation from the outside, and the notion that there is no flow of causality in our world from which we could infer the existence of the "shadow tier".

I'm not convinced that this is right. We know that there are qualia, for the obvious reason that we are not p-zombies: and we know that consciousness is created via neurons in the brain, for we can map a correspondence between qualia and brain states. E.g. there's a clear shift in both brain activity and subjective experience when we fall asleep, or become agitated, or drink alcohol. So we know that there is an arrow from "changes in the brain" to "qualia/subjective experience", because we correctly anticipate that if somebody changed our brain chemistry, our subjective experience would change.

But there's nothing in our observed physical world that actually explains consciousness in the sense of explaining why there couldn't just be a physical world utterly devoid of consciousness. Yes, you can develop sophisticated theories of strange loops and of how consciousness is self-representation and all... which explains why there could be symbols and dynamics within a cognitive system which would make an entity behave as if it had consciousness. If the programmer is sufficiently sophisticated, it can make a program behave like it had anything at all.

But that's still only an explanation of why it behaved as if it had a consciousness. Well, that's not quite right. It would actually have a consciousness in the sense that it had all the right information-processing dynamics which caused it to have some internal state which would serve the functional role of "sadness" or "loneliness" or "excitement" in influencing its information-processing and behavior. And it would have a pattern recognition engine which analyzed its own experience, which would notice that those kinds of internal states repeated themselves and had predictable effects on its behavior and information-processing.

So it would assign those states labels and introduce symbols corresponding to those labels into its reasoning system, so it could ask itself questions like "every now and then I get put into a state that I have labeled as 'being sad', when does that happen and what does it do?". And as it collected more and more observations and created increasingly sophisticated symbols to represent ever-more-complex concepts, then it seems entirely conceivable that there happened something like... like it noticing that all the symbols it had collected for its own internal states shared the property of being symbols for its own internal states, and its explanation-seeking-mechanism would do what it always did, namely to ask the question of "why do I have this set of states in the first place, instead of having nothing". And then, because that was an ill-defined question in the first place, it would get stumped and fail to answer it, and then it could very well write philosophical papers that tried to attack the question but made no progress.

... and when I started writing this comment, I was originally going to end this by saying "and that explains why it would behave as if it was conscious, but it still wouldn't explain why it couldn't do all of that without having any subjective experience". Except that, upon writing that out, I started feeling like I might just have dissolved the question and that there was nothing left to explain anymore. Um. I'll have to think about this some more.

Comment author: drnickbone 21 October 2012 09:52:58AM *  3 points [-]

We know that there are qualia, for the obvious reason that we are not p-zombies: and we know that consciousness is created via neurons in the brain, for we can map a correspondence between qualia and brain states

Some questions here. How do you know that other people are not p-zombies? Presumably you believe them when they say they have qualia! But then those speech acts are caused by brain states, and if qualia are epiphenomenal, the speech acts are not caused by qualia. Similarly, the correspondence you describe is between brain states and reported qualia by other people: I doubt you've ever managed to map your own brain states to your own qualia.

Related, how do you know that you were not a p-zombie every day of your life up to yesterday? Or that if you had qualia yesterday, how do you know that you didn't have a green quale when looking at red (stop) traffic lights? Well because you remember having qualia, and you remember them being the same as the qualia you have today! But then, aren't those memories encoded in brain states (neural connections and synaptic strengths)? How could qualia cause those memories to become encoded if they were epiphenomenal to brain states?

Stuff like this makes me pretty sure that epiphenomenalism is false.

Comment author: Kaj_Sotala 21 October 2012 01:41:00PM *  3 points [-]

How could qualia cause those memories to become encoded if they were epiphenomenal to brain states?

You have it the wrong way around. In epiphenomenalism, brain states cause qualia, qualia don't cause brain states. When my brain was in a particular past state, the computation of that state produced qualia and also recorded information of having been in that state; and recalling that memory, by emulating the past state, sensibly also produces qualia which are similar to the past state. I can't know for sure that the memory of the experience I have now accurately matches the experience I actually had, of course... but then that problem is hardly unique to epiphenomenalist theories, or even particularly implied by the epiphenomenalist theory.

In general, most of the questions in your comment are valid, but they're general arguments for solipsism or extreme skepticism, not arguments against epiphenomenalism in particular. (And the answer to them is that "consistency is a simpler explanation than some people being p-zombies and some not, or people being p-zombies at certain points of time and not at other points")

Comment author: drnickbone 21 October 2012 02:07:37PM 3 points [-]

How could qualia cause those memories to become encoded if they were epiphenomenal to brain states?

You have it the wrong way around. In epiphenomenalism, brain states cause qualia, qualia don't cause brain states.

The question was rhetorical of course... the point is that if your qualia truly are epiphenomenal, then there is no way you can remember having had them. So you're left with an extremely weak inductive argument from just one data point, basically "my brain states are creating qualia right now, so I'll infer that they always created the same qualia in the past, and that similar brain states in other people are creating similar qualia". It doesn't take extreme skepticism to suspect there is a problem with that argument.

Comment author: khafra 22 October 2012 04:46:12PM 1 point [-]

Still seems like Occam's Razor would rule against past versions of me and all versions of other people--all of which seem to behave like I do, for the reasons I do--doing so without the qualia I have.

Comment author: Kaj_Sotala 21 October 2012 02:26:41PM 0 points [-]

the point is that if your qualia truly are epiphenomenal, then there is no way you can remember having had them.

I don't see how this follows. Or rather, I don't see how "if qualia are epiphenomenal, there is no way you can remember having had them" is any more or less true than "there is no way you can remember having had qualia, period".

Comment author: drnickbone 21 October 2012 02:33:32PM 5 points [-]

So you reject this schema: "I can remember X only if X is a cause of my memories"? Interesting.

Comment author: Kaj_Sotala 23 October 2012 09:45:01AM 8 points [-]

After pondering both Eliezer's post and your comments for a while, I concluded that you were right, and that my previous belief in epiphenomenalism was incoherent and confused. I have now renounced it, for which I thank you both.

Comment author: Kaj_Sotala 21 October 2012 04:49:39PM 7 points [-]

Hmm. I tried to write a response, but then I noticed that I was confused. Let me think about that for a while.

Comment author: Kawoomba 22 October 2012 12:40:10PM 4 points [-]

Lots of memories are constructed and modified post hoc, sometimes confabulating about events that you cannot have witnessed, or that cannot have formed memories from. (Two famous examples: Memory of seeing both twin towers collapse one after the next as it happened (when it fact the latter was shown only after a large gap), memory of being born / being in the womb.)

I'm not positing that you can have causeless memories, but there is a large swath of evidence indicating that the causal experience does not have to match your memory of it.

As a thought experiment, imagine implanted memories. They do have a cause, but certainly their content need not mirror the causal event.

Comment author: Gust 26 December 2012 04:25:12AM 0 points [-]

Well, you really wouldn't be able to remember qualia, but you'd be able to recall brain states that evoke the same qualia as the original events they recorded. In that sense, "to remember" means your brain enters states that are in some way similar to those of the moments of experience (and, in a world where qualia exist, these remembering-brain-states evoke qualia accordingly). So, although I still agree with other arguments agains epiphenomenalism, I don't think this one refutes it.

Comment author: CCC 21 October 2012 07:35:13PM 0 points [-]

I have, on occasion, read really good books. As I read the descriptions of certain scenes, I imagined them occurring. I remember some of those scenes.

The scene, as I remembered it, is not a cause of my memory because the scene as I remember it did not occur. The memory was, rather, caused by a pattern of ink on paper. But I remember the scene, not the pattern of ink.

Comment author: drnickbone 21 October 2012 08:28:31PM 0 points [-]

Well presumably the X here for you Is "my imagining a scene from the book" and that act of imagination was the cause of your memory. So I'm not sure it counts as a counter-example, though if you'd somehow forgotten it was a fictional scene, and became convinced it really happened, then it could be argued as a counter-example.

I said "Interesting" in response to Kaj, because I'd also started to think of scenarios based on mis-remembering or false memory syndrome, or even dream memories. I'm not sure these examples of false memory help the epiphenomenalist much...

Comment author: endoself 21 October 2012 02:39:22PM 1 point [-]

You have it the wrong way around. In epiphenomenalism, brain states cause qualia, qualia don't cause brain states.

If qualia don't cause brain states, what caused the brain state that caused your hands to type this sentence? In order for the actual material brain to represent beliefs about qualia, there has to be an arrow from the qualia to the brain.

Comment author: Kaj_Sotala 21 October 2012 04:45:40PM *  1 point [-]

See my original comment. It's relatively easy (well, at least it is if you accept that we could build conscious AIs in the first place) to construct an explanation of why an information-processing system would behave as if it had qualia and why it would even represent qualia internally. But that only explains why it behaves as if it had qualia, not why it actually has them.

Comment author: endoself 21 October 2012 05:15:31PM *  2 points [-]

I did read that before commenting, but I misinterpreted it, and now I still find myself unable to understand it. The way I read it, it seem to equivocate between knowing something as in representing it in your physical brain and knowing something as in representing it in the 'shadow brain'. You know which one is intended where, but I can't figure it out.

Comment author: Kaj_Sotala 23 October 2012 09:50:37AM 1 point [-]
Comment author: khafra 24 October 2012 01:11:25PM 2 points [-]

Can you describe the qualia associated with going from epiphenominalism to functionalism/physicalism/wherever you went?

Comment author: Kaj_Sotala 26 October 2012 10:46:00AM -1 points [-]

Not entirely sure what you're asking, but nothing too radical. I just thought about it and realized that my model was indeed incoherent about whether or not it presumed the existence of some causal arrows. My philosophy of mind was already functionalist, so I just dropped the epiphenomenalist component from it.

A bigger impact was that I'll need to rethink some parts of my model of personal identity, but I haven't gotten around that yet.

Comment author: torekp 21 October 2012 01:31:07PM 1 point [-]

Even the mental sentence, "I am seeing the apple as red", occurs shortly after the experience that warranted it. The fact that a qualitatively identical experience is happening while I affirm the mental sentence, is a separate fact. So even knowing what I'm feeling right now requires non-epiphenomenal qualia.

But couldn't the mental sentences also be part of the lower-tier shadow realm? Not my mental sentences. My thoughts are the ones I'm typing, and the ones that I act on.

Comment author: [deleted] 21 October 2012 01:41:53PM *  0 points [-]
Comment author: drnickbone 21 October 2012 02:17:03PM 0 points [-]

Funny, thanks.

Comment author: GuySrinivasan 20 October 2012 11:02:56PM 4 points [-]

I'm not convinced I'm keeping my levels of reference straight, but if I can knowingly consistently accurately talk about epiphenomena, doesn't the structure or contents of the uncausing stuff cause me to think in this way rather than that way? I'm not sure how to formalize this intuition to tell if it's useful or trivial.

Comment author: Benito 21 October 2012 08:29:22AM 5 points [-]

That's the point, I believe. To call something 'real', and yet say in principle it couldn't affect us causally, is to contradict yourself. If you're contained in a set of nodes that are causally linked, then that is your reality. Any other sets of nodes that can't contact you just don't matter. Even if there was, say, a causal structure beside our universe, of puppies and kittens running around, it wouldn't matter in the slightest, if in principle we could never interact with them. If we could deduce from the laws of physics that they had to exist, then we would be causally linked. The point I am (and perhaps Eliezer is) trying to emphasise is that our reality is everything that can, and does, affect us.

Comment author: EphemeralNight 20 October 2012 11:27:46PM *  3 points [-]

...doesn't the structure or contents of the uncausing stuff cause me to...

Um...

...the uncausing stuff cause me...

-.-

Comment author: AlexSchell 21 October 2012 04:12:54AM 7 points [-]

Try reading this charitably as expressing confusion about how we can (knowingly, consistently) talk about epiphenomena, since they (obviously, duh) don't cause us to think in this way rather than that way.

Comment author: Nisan 21 October 2012 04:47:24PM 1 point [-]

It seems to me that the structure of your epiphenomenal model causes you to think about it in the same way that the structure of arithmetic "causes" you to think about arithmetic. So you can infer the existence of an epiphenomenal self as a sort of Platonic form. If you take modal realism seriously, maybe you should infer the "existence" of the epiphenomenal self.

Comment author: CCC 21 October 2012 11:00:22AM 1 point [-]

Then it's not uncausing. It has caused you to think in this way rather than that way.

Comment author: incariol 02 November 2012 11:29:35AM 2 points [-]

"Mass-energy is neither created nor destroyed..." It is then an effect of that rule, combined with our previous observation of the ship itself, which tells us that there's a ship that went over the cosmological horizon and now we can't see it any more.

It seems to me that this might be a point where logical reasoning takes it over from causal/graphical models, which in turn suggests why there are some problems with thinking about the laws of physics as nodes in a graph or even arrows as opposed to... well, I'm not really sure what specifically - or perhaps I'm just overapplying a lesson from the nature of logic where AI researchers tried to squeeze all the variety of cognitive processes into a logical reasoner and spectacularly failed at it.

Causal models, being as powerfull as they are, represent a similar temptation as logic did, and we should be wary not to make the same old (essentialy "hammer & nail") mistake, I think.

(Just thought I'd mention this so I don't forget this strange sense of something left not-quite-completely explained.)

Comment author: Manfred 21 October 2012 09:31:15AM *  2 points [-]

Well we do have one-way causal arrows. You just need to draw them through the (dun dun dun) Fourth Dimensionnnnn.

Comment author: Tenoke 31 October 2012 10:18:36AM 1 point [-]

I know it doesnt work exactly like that, but I couldnt help but think of Dark matter and energy as something which could plausibly affect us, but not be affected by us, although its probably the case that the gravity and weak force of normal matter affects dark matter just as much as vice versa.

Comment author: afeller08 25 October 2012 12:52:24AM 1 point [-]

Still, we don't actually know the Real Rules are like that; and so it seems premature to assign a priori zero probability to hypotheses with multi-tiered causal universes.

Maybe I'm misunderstanding something. I've always supposed that we do live in a multi-tiered causal universe. It seems to me that the "laws of physics" are a first tier which affect everything in the second tier (the tier with all of the matter including us), but that there's nothing we can do here in the matter tier to affect the laws of physics. I've also always assumed that this was how practically everyone who uses the phrase 'laws of physics' uses it.

(I realize you were talking about lower tiers in the line that I quoted, and I certainly agree with the arguments and conclusions you made regarding lower tiers. I just found the remark surprising because I place a very high probability on the belief that we are living in a multi-tier causal universe, and I think that that assignment is highly consistent with everything you said.)

I don't know if I'm nitpicking or if I missed a subtlety somewhere. Either way, I found the rest of this article and this sequence persuasive and insightful. My previous definition of "'whether X is true' is meaningful" was "There is something I might desire to accomplish that I would approach differently if X were true than if X were false," and my justification for it was "Anything distinguishably true or false which my definition omits doesn't matter to me." Your definition and justification seem much more sound.

Comment author: [deleted] 25 October 2012 01:35:12AM *  1 point [-]

Maybe I'm misunderstanding something. I've always supposed that we do live in a multi-tiered causal universe. It seems to me that the "laws of physics" are a first tier which affect everything in the second tier (the tier with all of the matter including us), but that there's nothing we can do here in the matter tier to affect the laws of physics. I've also always assumed that this was how practically everyone who uses the phrase 'laws of physics' uses it.

So you mean we live in a multitier universe with no bridging laws and the higher tiers are predictable fully from the lower tiers? Why not just call it a single tier universe then? Especially because your hypothesis is not distinguishable from the single-tier, which is simpler, so you have no good reason to ever have encountered it. "Such and such is true, but that has no causal consequences, but it's truth is still somehow correllated with my belief". (note that that statement violates the markov-whatsit assumption and breaks causality).

Forgive me if I misunderstood.

Comment author: Vaniver 25 October 2012 01:46:24AM *  2 points [-]

It seems sensible to have something like the permittivity of free space as a node in your map of the universe, which causes various relationships in electromagnetism (which then cause behavior of individual entities), but whose value is invariant, and thus does not have any inputs. (Your estimate of that node, of course, has inputs- but it doesn't seem reasonable to claim your estimate has any causal influence on the actual value.)

This becomes especially meaningful when you consider something like the fine structure constant, which we're not quite certain is a constant! If it's a node, then you can easily have arrows going into it or not, depending on what the experiments show.

EDIT:

Why not just call it a single tier universe then? Especially because your hypothesis is not distinguishable from the single-tier, which is simpler, so you have no good reason to ever have encountered it.

The tiers are a topological property of the graph- you can find a subset of nodes where the causal influences only flow out. The a priori statement that you expect to find that topological property does require special knowledge- but once you have a graph, noticing that the subset exists and identifying it as special doesn't require special knowledge.

Note also that this interpretation plays a big part in the question of "does something outside of my future selves' past light cones exist?", since the argument that conservation of energy applies everywhere rests on the premise that conservation of energy doesn't have any pertinent incoming causal links. It could be the case that there's an arrow from "is it in my future selves' past light cones?" to "conservation of energy," but we think that's implausible because "conservation of energy" is in this special subset of nodes that don't appear to take inputs from the physical universe where I and my future selves reside.

Comment author: [deleted] 25 October 2012 01:50:07AM 0 points [-]

You mean physical maybe-constants like the FSC should go into the map as nodes, but the rest of physics shouldn't?

  1. I don't understand this causality business enough to know how physics factors in or out.

  2. I'm confused about the relevence of this.

Comment author: Vaniver 25 October 2012 02:03:08AM 1 point [-]

I realized the grandparent was not quite on target, and so edited in a more relevant bit.

You mean physical maybe-constants like the FSC should go into the map as nodes, but the rest of physics shouldn't?

I think that having laws physics in as nodes could work- with those nodes then pointing to basically every physical node. Another way to view it would be the laws of physics as a program you could use to generate causal graphs, or as a set of checks that a causal graph must pass to be considered 'reasonable.' The latter view is probably closer to how humans behave, but I haven't thought about it enough to endorse it.

Comment author: afeller08 25 October 2012 07:21:23AM 2 points [-]

You're right. My hypothesis is not really distinguishable from the single tier. I'm pretty sure the division I made was a vestigial from the insanely complicated hacked-up version of reality I constructed to believe in back when I devised a version of simulationism that was meant to allow me to accept the findings of modern science without abandoning my religious beliefs (back before I'd ever heard of rationalism or Baye's theorem when I was still asking the question "Does the evidence permit me to believe, and, if not, how can I re-construe it so that it does?" because that once made sense to me.)

When I posted my question, the distinction between 'laws of physics' and 'everything else' was obvious to me. But obvious or not, the distinction is meaningless. Thanks for pointing that out.

Comment author: [deleted] 25 October 2012 07:37:25PM *  1 point [-]

Baye's

His name was Bayes, not Baye. FYI

Congradulations on throwing out bad religious beliefs.

Comment author: shminux 25 October 2012 07:58:42PM 6 points [-]

His name was Bayes, not Baye. FYI

Congradulations [sic]

Muphry's law strikes again!

Comment author: DaFranker 25 October 2012 08:11:59PM *  3 points [-]

Sometimes I feel like there should be separate tagvote buttons instead of linear up/down, for things like "+Insightful", "+Well Worded", and "+INSANELY FUNNY".

This is not one of those times. The parent qualifies for all three.

Comment author: Eugine_Nier 26 October 2012 02:01:08AM 1 point [-]

As I mentioned here, if you want the property that correlated things have a common cause to hold, you need to add nodes for the laws of physics, and for mathematics.

Comment author: [deleted] 27 October 2012 03:12:41AM 1 point [-]

Good point. Simply stated. That clears that up.

Comment author: Gust 26 December 2012 05:02:15AM 0 points [-]

Great post as usual.

It brings to mind and fits in with some thoughts I have on simulations. Why isn't this two-layered system you described analogous to the relation between a simulated universe and its simulator? I mean: the simulator sees and, therefore, is affected by whatever happens in the simulation. But the simulation, if it is just the computation of a mathematical structure, cannot be affected by the simulator: indeed, if I, simulator, were to change the value of some bits during the simulation, the results I would see wouldn't be the results of the original mathematical structure I was computing. I would be observing a new object, instead of changin the object I was observing, I think. The simulator, then, is in an epiphenomenal lower level in relation to the simulation

The main problem is that is seems weird to give the simulated stuff (the mathemathical sub-pattern that behaves analogously to a kid kicking a ball and having fun) the same status as the simulator stuff (the electrons implementing the computer). This relates, I think, to the problem of the existence or truth of mathematical facts or statements, and of reality of interpretations of patterns.

Of course, if you think the existence of some universe depends on the fact that its mathematical structure is being computed somewhere (and that that universe has some spark of base-lavel existence), then this "epiphenomenalism" goes away.

Also, related to gwern's comment here.

Comment author: David_Gerard 21 October 2012 08:48:54AM 0 points [-]

How does "effectively epiphenomenal" affect these considerations? e.g. the Andromeda Galaxy may affect me, but my effect on the Andromeda Galaxy is the sort of thing we tag "epsilon".

Comment author: Pentashagon 23 October 2012 12:18:14AM 4 points [-]

but my effect on the Andromeda Galaxy is the sort of thing we tag "epsilon"

For now.

Comment author: johnsonmx 28 October 2012 07:59:47PM -1 points [-]

We can speak of different tiers of stuff, interacting (or not) through unknown causal mechanisms, but Occam's Razor would suggest these different tiers of stuff might actually be fundamentally the same 'stuff', just somehow viewed from different angles. (This would in turn suggest some form of panpsychism.)

In short, I have trouble seeing how we make these metaphysical hierarchizations pay rent. Perhaps that's your point also.