Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Mixed Reference: The Great Reductionist Project

25 Post author: Eliezer_Yudkowsky 05 December 2012 12:26AM

Followup toLogical PinpointingCausal Reference

Take the universe and grind it down to the finest powder and sieve it through the finest sieve and then show me one atom of justice, one molecule of mercy.

- Death, in Hogfather by Terry Pratchett

Meditation: So far we've talked about two kinds of meaningfulness and two ways that sentences can refer; a way of comparing to physical things found by following pinned-down causal links, and logical validity by comparison to models pinned-down by axioms. Is there anything else that can be meaningfully talked about? Where would you find justice, or mercy?

... 
... 
...

Suppose that I pointed at a couple of piles of apples on a table, a pile of two apples and a pile of three apples.

And lo, I said:  "If we took the number of apples in each pile, and multiplied those numbers together, we'd get six."

Nowhere in the physical universe is that 'six' written - there's nowhere in the laws of physics where you'll find a floating six. Even on the table itself there's only five apples, and apples aren't fundamental. Or to put it another way:

Take the apples and grind them down to the finest powder and sieve them through the finest sieve and then show me one atom of sixness, one molecule of multiplication.

Nor can the statement be true as a matter of pure math, comparing to some Platonic six within a mathematical model, because we could physically take one apple off the table and make the statement false, and you can't do that with math.

This question doesn't feel like it should be very hard.  And indeed the answer is not very difficult, but it is worth spelling out; because cases like "justice" or "mercy" will turn out to proceed in a similar fashion.

Navigating to the six requires a mixture of physical and logical reference.  This case begins with a physical reference, when we navigate to the physical apples on the table by talking about the cause of our apple-seeing experiences:

Next we have to call the stuff on the table 'apples'.  But how, oh how can we do this, when grinding the universe and running it through a sieve will reveal not a single particle of appleness?

This part was covered at some length in the Reductionism sequence.  Standard physics uses the same fundamental theory to describe the flight of a Boeing 747 airplane, and collisions in the Relativistic Heavy Ion Collider.  Nuclei and airplanes alike, according to our understanding, are obeying special relativity, quantum mechanics, and chromodynamics.

We also use entirely different models to understand the aerodynamics of a 747 and a collision between gold nuclei in the RHIC.  A computer modeling the aerodynamics of a 747 may not contain a single token, a single bit of RAM, that represents a quark.  (Or a quantum field, really; but you get the idea.)

So is the 747 made of something other than quarks?  And is the statement "this 747 has wings" meaningless or false?  No, we're just modeling the 747 with representational elements that do not have a one-to-one correspondence with individual quarks.

Similarly with apples.  To compare a mental image of high-level apple-objects to physical reality, for it to be true under a correspondence theory of truth, doesn't require that apples be fundamental in physical law.  A single discrete element of fundamental physics is not the only thing that a statement can ever be compared-to.  We just need truth conditions that categorize the low-level states of the universe, so that different low-level physical states are inside or outside the mental image of "some apples on the table" or alternatively "a kitten on the table".

Now we can draw a correspondence from our image of discrete high-level apple objects, to reality.

Next we need to count the apple-objects in each pile, using some procedure along the lines of going from apple to apple, marking those already counted and not counting them a second time, and continuing until all the apples in each heap have been counted.  And then, having counted two numbers, we'll multiply them together.  You can imagine this as taking the physical state of the universe (or a high-level representation of it) and running it through a series of functions leading to a final output:

And of course operations like "counting" and "multiplication" are pinned down by the number-axioms of Peano Arithmetic:

And we shouldn't forget that the image of the table, is being calculated from eyes which are in causal contact with the real table-made-of-particles out there in physical reality:

And then there's also the point that the Peano axioms themselves are being quoted inside your brain in order to pin down the ideal multiplicative result - after all, you can get multiplications wrong - but I'm not going to draw the image for that one.  (We tried, and it came out too crowded.)

So long as the math is pinned down, any table of two apple piles should yield a single output when we run the math over it. Constraining this output constrains the possible states of the original, physical input universe:

And thus "The product of the apple numbers is six" is meaningful, constraining the possible worlds. It has a truth-condition, fulfilled by a mixture of physical reality and logical validity; and the correspondence is nailed down by a mixture of causal reference and axiomatic pinpointing.

I usually simplify this to the idea of "running a logical function over the physical universe", but of course the small picture doesn't work unless the big picture works.


The Great Reductionist Project can be seen as figuring out how to express meaningful sentences in terms of a combination of physical references (statements whose truth-value is determined by a truth-condition directly correspnding to the real universe we're embedded in) and logical references (valid implications of premises, or elements of models pinned down by axioms); where both physical references and logical references are to be described 'effectively' or 'formally', in computable or logical form.  (I haven't had time to go into this last part but it's an already-popular idea in philosophy of computation.)

And the Great Reductionist Thesis can be seen as the proposition that everything meaningful can be expressed this way eventually.

But it sometimes takes a whole bunch of work.

And to notice when somebody has subtly violated the Great Reductionist Thesis - to see when a current solution is not decomposable to physical and logical reference - requires a fair amount of self-sensitization before the transgressions become obvious.


Example:  Counterfactuals.

Consider the following pair of sentences, widely used to introduce the idea of "counterfactual conditioning":

  • (A) If Lee Harvey Oswald didn't shoot John F. Kennedy, someone else did.
  • (B) If Lee Harvey Oswald hadn't shot John F. Kennedy, someone else would've.

The first sentence seems agreeable - John F. Kennedy definitely was shot, historically speaking, so if it wasn't Lee Harvey Oswald it was someone.  On the other hand, unless you believe the Illuminati planned it all, it doesn't seem particularly likely that if Lee Harvey Oswald had been removed from the equation, somebody else would've shot Kennedy instead.

Which is to say that sentence (A) appears true, and sentence (B) appears false.

One of the historical questions about the meaning of causal models - in fact, of causal assertions in general - is, "How does this so-called 'causal' model of yours, differ from asserting a bunch of statistical relations?  Okay, sure, these statistical dependencies have a nice neighborhood-structure, but why not just call them correlations with a nice neighborhood-structure; why use fancy terms like 'cause and effect'?"

And one of the most widely endorsed answers, including nowadays, is that causal models carry an extra meaning because they tell us about counterfactual outcomes, which ordinary statistical models don't.  For example, suppose this is our causal model of how John F. Kennedy got shot:

Kennedy causes Oswald

Roughly this is intended to convey the idea that there are no Illuminati:  Kennedy causes Oswald to shoot him, does not cause anybody else to shoot him, and causes the Moon landing; but once you know that Kennedy was elected, there's no correlation between his probability of causing Oswald to shoot him and his probability of causing anyone else to shoot him.  In particular, there's no Illuminati who monitor Oswald and send another shooter if Oswald fails.

In any case, this diagram also implies that if Oswald hadn't shot Kennedy, nobody else would've, which is modified by a counterfactual surgery a.k.a. the do(.) operator, in which a node is severed from its former parents, set to a particular value, and its descendants then recomputed:

do Oswald=N

 

And so it was claimed that the meaning of the first diagram is embodied in its implicit claim (as made explicit in the second diagram) that "if Oswald hadn't shot Kennedy, nobody else would've".  This statement is true, and if all the other implicit counterfactual statements are also true, the first causal model as a whole is a true causal model.

What's wrong with this picture?

Well... if you're strict about that whole combination-of-physics-and-logic business... the problem is that there are no counterfactual universes for a counterfactual statement to correspond-to.  "There's apples on the table" can be true when the particles in the universe are arranged into a configuration where there's some clumps of organic molecules on the table.  What arrangement of the particles in this universe could directly make true the statement "If Oswald hadn't shot Kennedy, nobody else would've"?  In this universe, Oswald did shoot Kennedy and Kennedy did end up shot.

But it's a subtle sort of thing, to notice when you're trying to establish the truth-condition of a sentence by comparison to counterfactual universes that are not measurable, are never observed, and do not in fact actually exist.

Because our own brains carry out the same sort of 'counterfactual surgery' automatically and natively - so natively that it's embedded in the syntax of language.  We don't say, "What if we perform counterfactual surgery on our models to set 'Oswald shoots Kennedy' to false?"  We say, "What if Oswald hadn't shot Kennedy?"  So there's this counterfactual-supposition operation which our brain does very quickly and invisibly to imagine a hypothetical non-existent universe where Oswald doesn't shoot Kennedy, and our brain very rapidly returns the supposition that Kennedy doesn't get shot, and this seems to be a fact like any other fact; and so why couldn't you just compare the causal model to this fact like any other fact?

And in one sense, "If Oswald hadn't shot Kennedy, nobody else would've" is a fact; it's a mixed reference that starts with the causal model of the actual universe where there are actually no Illuminati, and proceeds from there to the logical operation of counterfactual surgery to yield an answer which, like 'six' for the product of apples on the table, is not actually present anywhere in the universe.  But you can't say that the causal model is true because the counterfactuals are true.  The truth of the counterfactuals has to be calculated from the truth of the causal model, followed by the implications of the counterfactual-surgery axioms.  If the causal model couldn't be 'true' or 'false' on its own, by direct comparison to the actual real universe, there'd be no way for the counterfactuals to be true or false either, since no actual counterfactual universes exist.


So that business of counterfactuals may sound like a relatively obscure example (though it's going to play a large role in decision theory later on, and I expect to revisit it then) but it sets up some even larger points.

For example, the Born probabilities in quantum mechanics seem to talk about a 'degree of realness' that different parts of the configuration space have (proportional to the integral over squared modulus of that 'world').

Could the Born probabilities be basic - could there just be a basic law of physics which just says directly that to find out how likely you are to be in any quantum world, the integral over squared modulus gives you the answer?  And the same law could've just as easily have said that you're likely to find yourself in a world that goes over the integral of modulus to the power 1.99999?

But then we would have 'mixed references' that mixed together three kinds of stuff - the Schrodinger Equation, a deterministic causal equation relating complex amplitudes inside a configuration space; logical validities and models; and a law which assigned fundamental-degree-of-realness a.k.a. magical-reality-fluid.  Meaningful statements would talk about some mixture of physical laws over particle fields in our own universe, logical validities, and degree-of-realness.

This is just the same sort of problem if you say that causal models are meaningful and true relative to a mixture of three kinds of stuff, actual worlds,  logical validities, and counterfactuals, and logical validities.  You're only supposed to have two kinds of stuff.

People who think qualia are fundamental are also trying to build references out of at least three different kinds of stuff: physical laws, logic, and experiences.

Anthropic problems similarly revolve around a mysterious degree-of-realness, since presumably when you make more copies of people, you make their experiences more anticipate-able somehow.  But this doesn't say that anthropic questions are meaningless or incoherent.  It says that since we can only talk about anthropic problems using three kinds of stuff, we haven't finished Doing Reductionism to it yet.  (I have not yet encountered a claim to have finished Reducing anthropics which (a) ends up with only two kinds of stuff and (b) does not seem to imply that I should expect my experiences to dissolve into Boltzmann-brain chaos in the next instant, given that if all this talk of 'degree of realness' is nonsense, there is no way to say that physically-lawful copies of me are more common than Boltzmann brain copies of me.)

Or to take it down a notch, naive theories of free will can be seen as obviously not-completed Reductions when you consider that they now contain physics, logic, and this third sort of thingy called 'choices'.

And - alas - modern philosophy is full of 'new sorts of stuff'; we have modal realism that makes possibility a real sort of thing, and then other philosophers appeal to the truth of statements about conceivability without any attempt to reduce conceivability into some mixture of the actually-physically-real-in-our-universe and logical axioms; and so on, and so on.

But lest you be tempted to think that the correct course is always to just envision a simpler universe without the extra stuff, consider that we do not live in the 'naive un-free universe' in which all our choices are constrained by the malevolent outside hand of physics, leaving us as slaves - reducing choices to physics is not the same as taking a naive model with three kinds of stuff, and deleting all the 'choices' from it.  This is confusing the project of getting the gnomes out of the haunted mine, with trying to unmake the rainbow.  Counterfactual surgery was eventually given a formal and logical definition, but it was a lot of work to get that far - causal models had to be invented first, and before then, people could only wave their hands frantically in the air when asked what it meant for something to be a 'cause'.  The overall moral I'm trying convey is that the Great Reductionist Project is difficult; it's not a matter of just proclaiming that there's no gnomes in the mine, or that rainbows couldn't possibly be 'supernatural'.  There are all sorts of statement that were not originally, or are presently not obviously decomposable into physical law plus logic; but that doesn't mean you just give up immediately.  The Great Reductionist Thesis is that reduction is always possible eventually.  It is nowhere written that it is easy, or that your prior efforts were enough to find a solution if one existed.

Continued next time with justice and mercy (or rather, fairness and goodness).  Because clearly, if we end up with meaningful moral statements, they're not going to correspond to a combination of physics and logic plus morality.


Mainstream status.

Part of the sequence Highly Advanced Epistemology 101 for Beginners

Next post: "By Which It May Be Judged"

Previous post: "Causal Universes"

Comments (353)

Comment author: TsviBT 05 December 2012 09:41:20AM *  16 points [-]

Not to be obnoxious, but...

You're only supposed to have two kinds of stuff.

Why two?

ETA: I feel like I may have distracted from the thrust of the post. I think the main point was that there really really probably shouldn't be more then two stuffs, which is legit.

Comment author: Armok_GoB 05 December 2012 04:32:02PM 11 points [-]

Because Tegmark 4 isn't mainstream enough yet to get it down to one.

If there is a way to reduce it to zero or not is one discovery I'm much looking forward to, but there probably isn't. It certainly seems totally impossible, but that only really means "I can't think of a way to do it".

Comment author: DanArmak 05 December 2012 07:57:09PM 4 points [-]

What would that mean? How do you reduce something to nothing? Or, well, everything to nothing?

Comment author: Eliezer_Yudkowsky 05 December 2012 11:11:25PM 5 points [-]

It does indeed seem possible that in the long run we'll end up with one kind of stuff, either from the reduction of logic to physics, or the reduction of physics to math. It's also worth noting that my present model does have magical-reality-fluid in it, and it's conceivable that this will end up not being reduced. But the actual argument is something along the lines of, "We got it down to two crisp things, and all the proposals for three don't have the crisp nature of the two."

Comment author: Eugine_Nier 06 December 2012 05:52:00AM 3 points [-]

I think you're going to have better luck figuring out how to make the third thing crisp than reducing it to the first two.

Comment author: MaoShan 09 December 2012 04:36:28AM 2 points [-]

That seems to me more like an irreducible string of methods of interpretation. You have physics, whether you like it or not. If you want to understand the physics, you need math. And to use the math, you need logic. Physics itself does not require math or logic. We do, if we want to do anything useful with it. So it's not so much "reducible" as it is "interpretable"--physics is such that turning it into a bunch of numbers and wacky symbols actually makes it more understandable. But to draw from your example, you can't have a physical table with physically infinite apples sitting on it. Yet you can do math with infinities, but all the math in the world won't put more apples on that table.

...and since when is two apples sitting next to each other a pile??

Comment author: Armok_GoB 07 December 2012 05:26:04PM 1 point [-]

I only see one crisp thing and one thing borrowing some of the crispness of the first thing but mostly failing, in your model.

Comment author: MrMind 07 December 2012 05:14:17PM 1 point [-]

Just as mental gymnastics, what if instead we would be able to reduce physics and logic to magical reality fluid? :)

Anyway, for the "logic from physics" camp the work of Valentin Turchin seems interesting (above all "The cybernetic foundation of mathematics"). Also of notice the recent foundational program called "Univalent foundation".

Comment author: Eugine_Nier 08 December 2012 04:18:53AM 1 point [-]

Just as mental gymnastics, what if instead we would be able to reduce physics and logic to magical reality fluid? :)

I don't think you can reduce logic to anything else, since you would need to use logic to perform the reduction.

Comment author: MrMind 09 December 2012 12:05:32AM 0 points [-]

Well, since nobody have done that yet, we cannot be sure, but for example a reduction of logic to physics could look like this: "for a system built on top of this set of physics laws, this is the set of logical system available to it", which would imply that all the axiomatic system we use are only those accessible via our laws of physics. For an extreme seminal example, Turing machine with infinite time have a very different notion of "effective procedure".

Comment author: Eugine_Nier 09 December 2012 12:17:13AM 2 points [-]

"for a system built on top of this set of physics laws, this is the set of logical system available to it"

How would one show the above, or even build up a system on top of physical laws without using logic?

Comment author: MrMind 10 December 2012 11:11:43AM 1 point [-]

How would one show the above

I have (at the moment) no idea.

or even build up a system on top of physical laws without using logic?

It's clear that such a demonstration needs to use some kind of logic, but I think that doesn't undermine the (possible) reduction: if you show that the (set of) logic available to a system depends on the physical laws, you have shown that our own logic is determined by our own laws. This would entail that (possibly) different laws would have granted us different logics. I'm fascinated for example by the fact that the concept of "second order arithmetical truth" (SOAT) is inacessible by effective finite computation, but there are space-times that allow for infinite computation (and so system inhabiting such a world could possibly grasp effectively SOATs).

Comment author: bryjnar 05 December 2012 11:06:05PM *  7 points [-]

This.

EY's made a kind of argument that you should have two kinds of stuff (although I still think the logical pinpointing stuff is a bit weak), but he seems to be proceeding as if he'd shown that that was exhaustive. For all the arguments he's given so far, this third post could have been entitled "Experiences: the Third Kind of Stuff", and it would be consistent with what he's already said.

So yeah, we need an argument for; "You're only supposed to have two kinds of stuff."

Comment author: Eugine_Nier 06 December 2012 06:00:10AM 3 points [-]

He may be overreacting against a strain in philosophy that seeks to reduce everything to experience. Similar to the way behaviorism was an overreaction against Freud.

Comment author: MrMind 07 December 2012 05:18:42PM *  1 point [-]

So yeah, we need an argument for; "You're only supposed to have two kinds of stuff."

I think the whole point of "the great reductionist project" is that we don't really have a sufficiency theorem, so we should treat "no more than two" as an empirical hypothesis and proceed to discover its truth by the methods of science.

Comment author: shminux 05 December 2012 11:25:00PM -2 points [-]

this third post could have been entitled "Experiences: the Third Kind of Stuff"

Not third, first. There are only two kinds of stuff, experiences and models. Separating physical models from logical is rather artificial, both are used to explain experiences.

Comment author: RobbBB 06 December 2012 05:17:54AM 0 points [-]

We only access models via experiences. If you aren't willing to reduce models to experiences, why are you willing to reduce the physical world of apples and automobiles to experiences? You're already asserting a kind of positivistic dualism; I see no reason not to posit a third domain, the physical, to correspond to our concrete experiences, just as you've posited a 'model domain' (cf. Frege's third realm) to correspond to our abstract experiences.

Comment author: MixedNuts 05 December 2012 11:01:44AM 5 points [-]

Agreed. The number two is ridiculous and can't exist. Once you allow stuff to have a physical kind and a logical kind, what's to stop you from adding other kinds like degree-of-realness and Buddha-nature?

OTOH, logical abstractions steadfastly refuse to be reduced to physics. There may be hope for the other way around, a solution to "Why does stuff exist?" that makes the universe somehow necessary. (Egan's "conscious minds find themselves" is cute but implies either chaotic observations or something to get the minds started.) But we can't be very optimistic.

Comment author: army1987 05 December 2012 04:16:04PM *  3 points [-]

That's Tegmark's Mathematical Universe Hypothesis, the best explanation I've seen of is Section 8.1 “Something for Nothing” in Good and Real by Gary Drescher.

Comment author: TsviBT 05 December 2012 04:10:23PM 1 point [-]

For math as mere physics, see Egan's Luminous.

Comment author: Armok_GoB 05 December 2012 04:33:53PM -2 points [-]

This problem is already solved, the answer is here: http://arxiv.org/abs/0704.0646

Comment author: MixedNuts 05 December 2012 05:57:44PM 3 points [-]

I don't get it. Okay, obviously our universe is a mathematical structure, that's why physics works. "All math is real" is seductive, but "All computable math is real, but there are no oracles" is just weird; why would you expect that without experimental evidence of Church-Turing?

The idea that since there are twice as many infinite strings containing "1010" than "10100", the former must exist twice as much as the latter nicely explains why our universe is so simple. But I'm not at all convinced that universes like ours with stable observers are simpler than pseudorandom generators that pop out Boltzmann brains.

Comment author: Armok_GoB 05 December 2012 06:57:45PM 1 point [-]

That all math is "real" in some sense you observe directly any time you do any. The insight is not that math is MORE real than previously thought, but just that there isn't some additional find of realness. Sort of, this is an oversimplification.

Also check out: http://lesswrong.com/lw/1zt/the_mathematical_universe_the_map_that_is_the/

Comment author: shminux 05 December 2012 07:38:36PM 2 points [-]

Also check out: http://lesswrong.com/lw/1zt/the_mathematical_universe_the_map_that_is_the/

That post is a confused jumble of multiple misinterpretations of the word "exist".

Comment author: MixedNuts 05 December 2012 07:58:16PM 1 point [-]

If all levels of the Turing hierarchy are about as real, it's extremely unlikely our universe is at level zero. Yet Church-Turing looks pretty solid.

Comment author: endoself 08 December 2012 02:53:47AM *  1 point [-]

Combine this with the simulation hypothesis; a universe can only simulate less computationally expensive universes. (Of course this is handwavy and barely an argument, but it's possible something stronger could be constructed along these lines. I do think that much more work needs to be done here.)

Comment author: DaFranker 05 December 2012 04:46:40PM 2 points [-]

My best vulgarization, which I hope not to be a rationalization (read: Looking for more evidence that it is!), is that Physical kinds of stuff are about what is, while logical kinds of stuff are about "what they do".

If you have one lone particle¹ in an empty universe, there's only the one kind, the physical. The particle is there. Once you have two particles, the physical kind of stuff is about how they are, their description, while the logical stuff is about the axiom "these two particles interact" - and everything that derives from there, such as "how" they interact².

I do not see any room for more kinds of stuff that is necessary in order to fully and perfectly simulate all the states of the entire universe where these two particles exist. I also don't see how adding more particles is going to change that in any manner. As per the evidence we have, it seems extremely likely that our own universe is a version of this universe with simply more particles in it.

So really, you can reduce it to "one", if you're willing to hyper-reduce the conceptual fundamental "is" to the simple logical "do" - if you posit that a single particle in a separate universe simply does not exist, because the only existence of a particle is its interaction, and therefore interactions are the only thing that do exist. Then the distinction between the physical and logical becomes merely one of levels of abstraction, AFAICT, and can theoretically be done away with. However, the physical-logical two-rule seems to be useful, and the above seems extremely easy to misinterpret or confuse with other things.

  1. Defined as whatever is the most fundamentally reduced smallest possible unit of the universe, be that a point in a wave field equation, a quark, or anything else reality runs on.

  2. I've read some theories (and thought some of my own) implying that there is no real "how" of interaction, and that all the interactions are simply the simplest, most primitive possible kind of logical interaction, the reveal-existence function or something similar, and that from this function derive as abstractions all the phenomena we observe as "forces" or "kinds of interactions" or "transmissions of information". However, all such theories I've read are incomplete and also lack experimental verifiability. They do sound much simpler and more elegant, though.

Comment author: Peterdjones 05 December 2012 04:16:06PM 3 points [-]

How does EY know there are only two? Is it aprori knowledge? Is it empirical? Is it subject to falsification? How many failed reduictions-to-two-kinds-of-stuff do there have to be before TKoS is falsified?

Comment author: Eliezer_Yudkowsky 05 December 2012 11:08:14PM 0 points [-]

Always two there are. No more. No less.

Comment author: dspeyer 05 December 2012 04:46:38AM 15 points [-]

How confident are we in the Great Reductionist Thesis? Short of the Great Reductionist Project's success, what would be evidence for or against it?

Comment author: Eliezer_Yudkowsky 05 December 2012 09:24:42PM 0 points [-]

After it's been right the last 300 times or so, we should assess a substantial probability that it will be wrong before the 1,000th occasion, but believe much more strongly that it will be correct on the next occasion.

Comment author: JoshuaZ 05 December 2012 09:27:56PM *  4 points [-]

That doesn't seem to answer dspeyer's questions.

Comment author: Eliezer_Yudkowsky 05 December 2012 11:21:52PM 8 points [-]

Okay. I'll bet with somewhere around 50% probability that the Great Reductionist Project as I've described it works, with reduction to a single thing counting as success, and requiring magical reality-fluid counting as failure. I'll bet with 95% probability that it's right on the next occasion for anthropics and magical reality-fluid, and with 99+ probability that it's right on the next occasion for things that confuse me less; except that when it comes to e.g. free will, I don't know who I'd accept as a judge that didn't think the issue already settled.

Comment author: JoshuaZ 06 December 2012 05:00:56AM 4 points [-]

I'll bet with 95% probability that it's right on the next occasion for anthropics and magical reality-fluid,

Can you expand on what you mean by this?

Comment author: [deleted] 06 December 2012 01:56:00AM *  3 points [-]

Either the Great Reductionist Thesis ("everything meaningful can be expressed by [physics+logic] eventually") is itself expressible with physics+logic (eventually) or it isn't. If it is, then it might be true.

If it isn't, then the great reductionist thesis is not true, because the proposition it expresses is not meaningful. I'm worried about this possibility because the phrase 'everything meaningful' strikes me as dangerously self-referential.

Comment author: Eliezer_Yudkowsky 08 December 2012 05:58:11PM 21 points [-]

This is a reply to the long conversation below between Esar and RobbBB.

Let me first say that I am grateful to Esar and RobbBB for having this discussion, and double-grateful to RobbBB for steelmanning my arguments in a very proper and reasonable fashion, especially considering that I was in fact careless in talking about "meaningful propositions" when I should've remembered that a proposition, as a term of art in philosophy, is held to be a meaning-bearer by definition.

I'm also sorry about that "is meaningless is false" phrase, which I'm certain was a typo (and a very UNFORTUNATE typo) - I'm not quite sure what I meant by it originally, but I'm guessing it was supposed to be "is meaningless or false", though in the context of the larger debate now that I've read it, I would just say "colorless green ideas sleep furiously" is "meaningless" rather than false. In a strict sense, meaningless utterances aren't propositions so they can't be false. In a looser sense, an utterance like "Maybe we're living in an inconsistent set of axioms!" might be impossible to render coherent under strict standards of meaning, while also being colloquially called 'false' meaning 'not actually true' or 'mistaken'.

I'm coming at this from a rather different angle than a lot of existing philosophy, so let me do my best to clarify. First, I would like to distinguish the questions:

R1) What sort of things can be real?

R2) What thoughts do we want an AI to be able to represent, given that we're not certain about R1?

A (subjectively uncertain probabilistic) answer to R1 may be something like, "I'm guessing that only causal universes can be real, but they can be continuous rather than discrete, and in that sense aren't limited to mathematical models containing a finite number of elements, like finite Life boards."

The answer to R2 may be something like, "However, since I'm not sure about R1, I would also like my AI to be able to represent the possibility of a universe with Time-Turners, even though, in this case, the AI would have to use some generalization of causal reference to refer to the things around it, since it wouldn't live in a universe that runs on Pearl-style causal links."

In the standard sense of philosophy, question R2 is probably the one about 'meaning' or which assertions can be 'meaningful', although actually the amount of philosophy done around this is so voluminous I'm not sure there is a standard sense of 'meaning'. Philosophers sometimes try to get mileage out of claiming things are 'conceivable', e.g., the philosophical catastrophe of the supposed conceivability of P-zombies, and I would emphasize even at this level that we're not trying to get R1-mileage out of things being in R2. For example, there's no rule following from anything we've said so far that an R2-meaningful statement must be R1-possible, and to be particular and specific, wanting to conservatively build an AI that can represent Conway's Game of Life + Time-Turners, still allows us to say things like, "But really, a universe like that might be impossible in some basic sense, wihch is why we don't live there - to speak of our possibly living there may even have some deeply buried incoherence relative to the real rules for how things really have to work - but since I don't know this to be true, as a matter of my own mere mental state, I want my AI to be able to represent the possibility of time-travel." We might also imagine that a non-logically-omniscient AI needs to have an R2 which can contain inconsistent axiom sets the AI doesn't know to be inconsistent.

For things to be in R2, we want to show how a self-modifying AI could carry out its functions while having such a representation, which includes, in particular, being able to build an offspring with similar representations, while being able to keep track of the correspondence between those offspring's quoted representations and reality. For example, in the traditional version of P-zombies, there's a problem with 'if that was true, how could you possibly know it?' or 'How can you believe your offspring's representation is conjugate to that part of reality, when there's no way for it to maintain a correspondence using causal references?' This is the problem of a SNEEZE_VAR in the Matrix where we can't talk about whether its value is 0 or 1 because we have no way to make "0" or "1" refer to one binary state rather than the other.

Since the problems of R2 are the AI-conjugates of problems of reference, designation, maintainance of a coherent correspondence, etcetera, they fall within the realm of problems that I think traditional philosophy considers to be problems of meaning.

I would say that in human philosophy there should be a third issue R3 which arises from our dual desire to:

  • Not do that awful thing wherein somebody claims that only causal universes can be real and therefore your hypotheses about Time-Turners are meaningless noises.
  • Not do that awful thing wherein somebody claims that since P-zombies are "conceivable" we can know a priori that consciousness is a non-physical property.

In other words, we want to avoid the twin errors of (1) preemptively shooting down somebody who is making an honest effort to talk to us by claiming that all their words are meaningless noises, and (2) trying to extract info about reality just by virtue of having an utterance admitted into a debate, turning a given inch into a taken mile.

This leads me to think that human philosophers should also have a third category R3:

R3) What sort of utterances can we argue about in English?

which would roughly represent what sort of things 'feel meaningful' to a flawed human brain, including things like P-zombies or "I say that God can make a rock so heavy He can't lift it, and then He can lift it!" - admitting something into R3 doesn't mean it's logically possible, coherent, or 'conceivable' in some rigorous sense that you could then extract mileage from, it just means that we can go on having a conversation about it for a while longer.

When somebody comes to us with the P-zombie story, and claims that it's "conceivable" and they know this on account of their brain feeling able to conceive it, we want to reply, "That's what I would call 'arguable' (R3) and if you try to treat your intuitions about arguability as data, they're only directly data about which English sentences human brains can affirm. If you want to establish any stronger sense of coherence that you could get mileage from, such as coherence or logical possibility or reference-ability, you'll have to argue that separately from your brain's direct access to the mere affirmability of a mere English utterance."

At the same time, you're not shoving them away from the table like you would "colorless green ideas sleep up without clam any"; you're actually going to have a conversation about P-zombies, even though you think that in stricter senses of meaning like R2, the conversation is not just false but meaningless. After all, you could've been wrong about that nonmembership-in-R2 part, and they might be about to explain that to you.

The Great Reductionist Thesis is about R1 - the question of what is actually real - but it's difficult to have something that lies in a reductionist's concept of a strict R2, turn out to be real, such that the Great Reductionist Thesis is falsified. For example, if we think R1 is about causal universes, and then it turns out we're in Timetravel Life, the Great Reductionist Thesis has been confirmed, because Timetravel Life still has a formal logical description. Just about anything I can imagine making a Turing-computable AI refer to will, if real, confirm the Great Reductionist Thesis.

So is GRT philosophically vacuous from being philosophically unfalsifiable? No: to take an extreme case, suppose we have an uncomputable and non-logically-axiomatizable sensus divinatus enabling us to directly know God's existence, and by baptizing an AI we could give it this sensus divinatus in some way integrated into the rest of its mind, meaning that R2, R1, and our own universe all include things referrable-to only by a sensus divinatus. Then arguable utterances along the lines of, "Some things are inherently mysterious", would have turned out, not just to be in R2, but to actually be true; and the Great Reductionist Thesis would be false - contrary to my current belief that such utterances are not only colloquially false, but even meaningless for strict senses of meaning. But one is not licensed to conclude anything from my having allowed a sensus divinatus to be a brief topic of conversation, for by that I am not committing to admitting that it was strictly meaningful under strong criteria such as might be proposed for R2, but only that it stayed in R3 long enough for a human brain to say some informal English sentences about it.

Does this mean that GRT itself is merely arguable - that it talks about an argument which is only in R3? But tautologies can be meaningful in GRT, since logic is within "physics + logic". It looks to me like a completed theory of R2 should be something like a logical description of a class of universes and a class of representations corresponding to them, which would itself be in R2 as pure math; and the theory-of-R1 "Reality falls within this class of universes" could then be physically true. However, many informal 'negations' of R2 like "What about a sensus divinatus?" will only be 'arguable' in a human R3, rather than themselves being in R2 (as one would expect!).

Comment author: RobbBB 08 December 2012 08:33:26PM *  7 points [-]

R3) "What sort of utterances can we argue about in English?" is (perhaps deliberately) vague. We can argue about colorless green ideas, if nothing else at the linguistic level. Perhaps R3 is not about meaning, but about debate etiquette: What are the minimum standards for an assertion to be taken seriously as an assertion (i.e., not as a question, interjection, imperative, glossolalia, etc.). In that case, we may want to break R3 down into a number of sub-questions, since in different contexts there will be different standards for the admissibility of an argument.

I'm not sure what exactly a sensus divinatus is, or why it wouldn't be axiomatizable. Perhaps it would help flesh out the Great Reductionist Thesis if we evaluated which of these phenomena, if any, would violate it:

  1. Objective fuzziness. I.e., there are entities that, at the ultimate level, possess properties vaguely; perhaps even some that exist vaguely, that fall in different points on a continuum from being to non-being.

  2. Ineffable properties, i.e., ones that simply cannot be expressed in any language. The specific way redness feels to me, for instance, might be a candidate for logico-physical inexpressibility; I can perhaps ostend the state, but any description of that state will underdetermine the precise feeling.

  3. Objective inconsistencies, i.e., dialetheism. Certain forms of perspectivism, which relativize all truths to an observer, might also yield inconsistencies of this sort. Note that it is a stronger claim to assert dialetheism (an R1-type claim) than to merely allow that reasoning non-explosively with apparent contradictions can be very useful (an R2-type claim, affirming paraconsistent logics).

  4. Nihilism. There isn't anything.

  5. Eliminativism about logic, intentionality, or computation. Our universe lacks logical structure; basic operators like 'and' and 'all' and 'not' do not carve at the joints. Alternatively, the possibility of reference is somehow denied; AIs cannot represent, period. This is perhaps a stronger version of 2, on which everything, in spite of its seeming orderliness, is in some fashion ineffable.

Are these compatible with GRT? What else that we can clearly articulate would be incompatible? What about a model that is completely expressible in classical logic, but that isn't ontologically 'made of logic,' or of physics? I intuit that a classically modelable universe that metaphysically consists entirely of mind-stuff (no physics-stuff) would be a rather severe break from the spirit of reductive physicalism. But perhaps you intended GRT to be a much more modest and accommodating claim than everyday scientific materialism.

Comment author: Eliezer_Yudkowsky 08 December 2012 08:52:51PM 5 points [-]

I have no objection to your description of R3 - basically it's there so that (a) we don't think that something not immediately obviously being in R2 means we have to kick it off the table, and (b) so that when somebody claims their imagination is giving them veridical access to something, we can describe the thing accessed as membership in R3, which in turn is (and should be) too vague for anything else to be concluded thereby; you shouldn't be able to get info about reality merely by observing that you can affirm English utterances.

Insofar as your GRT violations all seem to me to be in R3 and not R2 (i.e., I cannot yet coherently imagine a state of affairs that would make them true), I'm mostly willing to agree that reality actually being that way would falsify GRT and my proposed R2. Unless you pick one of them and describe what you mean by it more exactly - what exactly it would be like for a universe to be like that, how we could tell if it were true - in which case it's entirely possible that this new version will end up in the logic-and-physics R2, and for similar reasons, wouldn't falsify GRT if true. E.g., a version of "nihilism" that is cashed out as "there is no ontologically fundamental reality-fluid", denial of "reference" in which there is no ontologically basic descriptiveness, eliminativism about "logic" which still corresponds to a computable causal process, "relativized" descriptions along the lines of Special Relativity, and so on.

This isn't meant to sneak reductionism in sideways into universes with genuinely ineffable magic composed of irreducible fundamental mental entities with no formal effective description in logic as we know it. Rather, it reflects the idea that even in an intuitive sense, sufficiently effable magic tends toward science, and since our own brains are in fact computable, attempts to cash out the ineffable in greater detail tend to turn it effable. The traditional First-Cause ontologically-basic R3 "God" falsifies reductionism; but if you redefine God as a Lord of the Matrix, let alone as 'natural selection', or 'the way things are', it doesn't. An irreducible soul falsifies GRT, until I interrogate you on exactly how that soul works and what it's made of and why there's still such a thing as brain damage, in which case my interrogation may cause you to adjust your claim and adjust it some more and finally end up in R2 (or even end up with a pattern theory of identity). It should also be noted that while the adjective "effable" is in R2, the adjective "ineffable" may quite possibly be in R3 only (can you exhibit an ineffable thing?)

I intuit that a classically modelable universe that metaphysically consists entirely of mind-stuff (no physics-stuff)

What does it mean to consist entirely of mind-stuff when all the actual structure of your universe is logical? What is the way things could be that would make that true, and how could we tell? This utterance is not yet clearly in my R2, which doesn't have anything in it to describe "metaphysically consists of'". (Would you consider "The substance of the cracker becomes the flesh of Christ while its accidents remain the same" to be in your equivalent of R2, or only in your equivalent of R3?)

PS: I misspelled it, it's http://en.wikipedia.org/wiki/Sensus_divinitatis

Comment author: RobbBB 09 December 2012 12:14:29AM *  5 points [-]

Here are three different doctrines:

  1. Expressibility. Everything (or anything) that is the case can in principle be fully expressed or otherwise represented. In other words, an AI is constructible-in-principle that could model every fact, everything that is so. Computational power and access-to-the-data could limit such an AI's knowledge of reality, but basic effability could not.

  2. Classical Expressibility. Everything (or anything) that is the case can in principle be fully expressed in classical logic. In addition to objective ineffability, we also rule out objective fuzziness, inconsistency, or 'gaps' in the World. (Perhaps we rule them out empirically; we may not be able to imagine a world where there is objective indeterminacy, but we at least intuit that our world doesn't look like whatever such a world would look like.)

  3. Logical Physicalism. The representational content of every true sentence can in principle be exhaustively expressed in terms very similar to contemporary physics and classical logic.

Originally I thought that your Great Reductionist Thesis was a conjunction of 1 and 3, or of 2 and 3. But your recent answers suggest to me that for you GRT may simply be Expressibility (1). Irreducibly unclassical truths are ruled out, not by GRT, but by the fact that we don't seem to need to give up principles like Non-Contradiction and Tertium Non Datur in order to Speak Every Truth. And mentalistic or supernatural truths are excluded only insofar as they violate Expressibility or just appear empirically unnecessary.

If so, then we should be very careful to distinguish your confidence in Expressibility from your confidence in physicalism. Neither, as I formulated them above, implies the other. And there may be good reason to endorse both views, provided we can give more precise content to 'terms very similar to contemporary physics and classical logic.' Perhaps the easiest way to give some meat to physicalism would be to do so negatively: List all the clusters that do seem to violate the spirit of physicalism. For instance:

  • mental (perspectival, 'subjective,' qualia-laden...) facts that cannot be fully expressed in non-mental terms.
  • otherwise anthropocentric (social, cultural, linguistic...) facts that cannot be fully expressed in non-anthropocentric terms.
  • spatiotemporal events without spatiotemporal causes
  • spatiotemporal events without spatiotemporal effects
  • abstract (non-spatiotemporal) objects that have causes
  • abstract objects that have effects
  • (perhaps) ineffable properties or circumstances

A list like this would give us some warning signs that a view, even if logically specifiable, may be deviating sharply from the scientific project. If you precisely stipulated in logical terms how Magic works, for instance, but its mechanism was extremely anthropocentric (e.g., requiring that Latin-language phonemes 'carve at the joints' of fundamental reality), that would seem to violate something very important about reductive physicalism, even if it doesn't violate Expressibility (i.e., we could program an AI to model magical laws of this sort).

What does it mean to consist entirely of mind-stuff when all the actual structure of your universe is logical?

I'm not sure what you mean by 'actual structure.' I would distinguish the Tegmark-style thesis 'the universe is metaphysically made of logic-stuff' from the more modest thesis 'the universe is exhaustively describable using purely logical terms.' If we learned that all the properties of billiard balls and natural numbers are equally specifiable in set-theoretic terms, I think we would still have at least a little more reason to think that numbers are sets than to think that billiard balls are sets.

So suppose we found a way to axiomatize 'x being from the perspective of y,' i.e., a thought and its thinker. If we (somehow) learned that all facts are ultimately and irreducibly perspectival (i.e., they all need an observer-term to be saturated), that might not contradict the expressibility thesis, but I think it would violate the spirit of physicalism.

(Would you consider "The substance of the cracker becomes the flesh of Christ while its accidents remain the same" to be in your equivalent of R2, or only in your equivalent of R3?)

I'm not sure. I doubt our universe has 'substance-accident' structure, but there might be some negative way to R2ify transubstantiation, even if (like epiphenomenalism or events-outside-the-observable-universe) it falls short of verifiability. Could we coherently model our universe as a byproduct of a cellular automaton, while lacking a way to test this model? If so, then perhaps we could model 'substance-properties' as unobservables that are similarly Behind The Scenes, but are otherwise structurally the same as accidents (i.e., observables).

Comment author: Eliezer_Yudkowsky 09 December 2012 03:48:50AM 4 points [-]

So... in my world, transubstantiation isn't in R2, because I can't coherently conceive of what a substance is, apart from accidents. For a similar reason, I don't yet have R2-language for talking about a universe being metaphysically made of anything. I mean, I can say in R3 that perhaps physics is made of cheese, just like I can say that the natural numbers are made of cheese, but I can't R2-imagine a coherent state of affairs like that. A similar objection applies to a logical universe which is allegedly made out of mental stuff. I don't know how to imagine a logically structured universe being made of anything.

Having Latin-language phonemes carve at the joints of fundamental reality seems very hard, because in my world Latin-language phonemes are already reduced - there's already sequential sound-patterns making them up, and the obvious way to have a logic describing the physics of such a world is to have complex specifications of the phonemes which are 'carving at the joints'. It's not totally clear to me how to make this complex thing a fundamental instead, though perhaps it could be managed via a logic containing enough special symbols - but to actually figure out how to write out that logic, you would have to use your own neuron-composed brain in which phonemes are not fundamental.

I do agree that - if it were possibly to rule out the Matrix, I mean, if spells not only work but the incantation is "Stupefy" then I know perfectly well someone's playing an S-day prank on me - that finding magic work would be a strong hint that the whole framework is wrong. If we actually find that prayers work, then pragmatically speaking, we've received a hint that maybe we should shut up and listen to what the most empirically powerful priests have to say about this whole "reductionism" business. (I mean, that's basically why we're listening to Science.) But that kind of meta-level "no, you were just wrong, shut up and listen to the spiritualist" is something you'd only execute in response to actually seeing magic, not in response to somebody hypothesizing magic. Our ability to hypothesize certain situations that would pragmatically speaking imply we were probably wrong about what was meaningful, doesn't mean we're probably wrong about what's meaningful. More along the lines of, "Somebody said something you thought was in R3(only), but they generated predictions from it and those predictions came true so better rethink your reasons for thinking it couldn't go in R2."

With all that said, it seems to me that R3-possibilities falsifying 1, 2, or (a generalization of 3 to other effectively or formally specified physics (e.g. Time-Turners)), and with the proviso that we're dealing in second-order logic rather than classical first-order logic, all seem to me to pretty much falsify the Great Reductionist Thesis. Some of your potential examples look to me like they're not in my R2 (e.g. mental facts that can't be expressed in non-mental terms) though I'm perfectly willing to discuss them colloquially in R3, and others seem relatively harmless (effects which aren't further causes of anything? I could write a computer program like that). I am hard-pressed to R2-meaningfully describe a state of affairs that falsifies R1, though I can talk about it in R3.

I have an overall agenda of trying to think like reality which says that I want my R1 to look as much like the universe as possible, and it's okay to contemplate restrictions which might narrow my R2 a lot relative to someone's R3, e.g. to say, "I can't seem to really conceive of a universe with fundamentally mental things anymore, and that's a triumph". So a lot of what looked to me years ago like meaningful non-reductionism, now seems more like meaningless non-reductionism relative to my new stricter conceptions of meaning - and that's okay because I'm trying to think less like a human and more like reality.

Comment author: [deleted] 09 December 2012 12:49:17AM *  2 points [-]

Can I run something by you? An argument occurred to me today that seems suspect, but I don't know what I'm getting wrong. The conclusion of the argument is that GRTt entails GRTm. For the purposes of this argument, GRTt is the statement that all true statements have a physico-logical expression (meaning physical, logical, or physical+logical expression). GRTm is the statement that all true and all false statements have a physico-logical expression.

P1) All true statements have a physico-logical expression. (GRTt)

P2) The negation of any false statement is true.

P3) If a statement has a physico-logical expression, its negation has a physico-logical expression.

P4) All false statements have a physico-logical expression.

C) All true and all false statements have a physical-logical expression. (GRTm)

So for example, suppose XYZ is false, and has no physico-logical expression. If XYZ is false, then ~XYZ is true. By GRTt, ~XYZ has a physico-logical expression. But if ~XYZ has a physico-logical expression, then ~(~XYZ), or XYZ, does. Throwing a negation in front of a statement can't change the nature of the statement qua reducible.

Therefore, GRTt entails GRTm. What do you think?

Comment author: RobbBB 06 December 2012 05:10:42AM *  2 points [-]

I don't see anything wrong with this kind of self-reference. We can only explain what generalizations are by asserting generalizations about generalization; but that doesn't undermine generalization itself. GRT would only be an immediate problem for itself if GRT didn't encompass itself.

Comment author: [deleted] 06 December 2012 06:00:10PM 1 point [-]

Okay, so lets assume that the generalization side of things is not a problem, though I hope you'll grant me that if a generalization about x's is meaningful, propositions expressing x's individually are meaningful. That is, if 'every meaningful proposition can be expressed by physics+logic (eventually)', then 'the proposition "the cat is on the mat" is meaningful' is meaningful. It's this that I'm worried about, and the generalization only indirectly. So:

1) A proposition is meaningful if and only if it is expressible by physics+logic, or merely by logic.

2) If a proposition is expressible by physics+logic, it constrains the possible worlds.

3) If the proposition "the cat is on the mat" is meaningful, and it is expressible by physics+logic, then it constrains the possible worlds.

4) If the proposition "the cat is on the mat" constrains the possible worlds, then the proposition "the proposition 'the cat is on the mat' is meaningful" does not constrain the possible worlds. Namely, no proposition of the form '"XYZ" constrains the possible worlds' itself constrains the possible worlds.

So if 'XYZ' constrains the possible worlds, then for every possible world, XYZ is either true of that world or false of that world. But if the proposition '"XYZ" constrains the possible worlds' expresses simply that, namely that for every possible world XYZ is either true or false of that world, then there is no world of which '"XYZ" constrains the possible worlds' is false.

5) The proposition 'the proposition "the cat is on the mat" is meaningful' is not both meaningful and expressible by physics+logic. But it is meaningful, and therefore (as per premise 1) it is expressible by mere logic.

6) Every generalization about a purely logical claim is itself a purely logical claim (I'm not sure about this premise)

7) The GRT is a purely logical claim.

I'm thinking EY wants to get off the GRT boat here: I don't think he intends the GRT to be a logical axiom or derivable from logical axioms. Nevertheless, if he does want the GRT to be an axiom of logic, and in order for it to be a meaningful axiom of logic, it still has to pick out one logical model as opposed to another.

But here, the problem simply recurs. If 'The proposition 'GRT' is meaningful' is meaningful then it doesn't, in the relevant respect, pick out one logical model as opposed to another.

Does that make sense?

Comment author: RobbBB 06 December 2012 06:26:41PM *  2 points [-]

2) If a proposition is expressible by physics+logic, it constrains the possible worlds.

I don't think we need this rule. It would make logical truths / tautologies meaningless, inexpressible, or magical. (We shouldn't dive into Wittgensteinian mysticism that readily.)

4) If the proposition "the cat is on the mat" constrains the possible worlds, then the proposition "the proposition 'the cat is on the mat' is meaningful" does not constrain the possible worlds.

That depends on what you mean by "proposition." The written sentence "the cat is on the mat" could have been ungrammatical or semantically null, like "colorless green ideas sleep furiously." After all, a different linguistic community could have existed in the role of the English language. So our semantic assertion could be ruling out worlds where "the cat is on the mat" is ill-formed.

On the other hand, if by "proposition" you mean "the specific meaning of a sentence," then your sentence is really saying "the meaning of 'the cat is on the mat' is a meaning," which is just a special case of the tautology "meanings are meanings." So if we aren't committed to deeming tautologies meaningless in the first place, we won't be committed to deeming this particular tautology meaningless.

But if the proposition '"XYZ" constrains the possible worlds' expresses simply that, namely that for every possible world XYZ is either true or false of that world, then there is no world of which '"XYZ" constrains the possible worlds' is false.

This looks like a problem of self-reference, but it's really a problem of essence-selection. When we identify something as 'the same thing' across multiple models or possible worlds, we're stipulating an 'essence,' a set of properties providing identity-conditions for an object. Without such a stipulation, we couldn't (per Leibniz's law) identify objects as being 'the same' while they vary in temporal, spatial, or other properties. If we don't include the specific meaning of a sentence in its essence, then we can allow that the 'same' sentence could have had a different meaning, i.e., that there are models in which sentence P does not express the semantic content 'Q.' But if we instead treat the meaning of P as part of what makes a sentence in a given model P, then it is contradictory to allow the possibility that P would lack the meaning 'Q,' just as it would be contradictory to allow the possibility that P could have existed without P existing.

What's important to keep in mind is that which of these cases arises is a matter of our decision. It's not a deep metaphysical truth that some essences are 'right' and some are 'wrong;' our interests and computational constraints are all that force us to think in terms of essential and inessential properties at all.

If 'The proposition 'GRT' is meaningful' is meaningful then it doesn't, in the relevant respect, pick out one logical model as opposed to another.

Only because you've stipulated that meaningfulness is essential to GRT (and to propositions in general). This isn't a spooky problem; you could have generated the same problem by claiming that 'all cats are mammals' fails to constrain the possible worlds, on the grounds that cats are essentially mammals, i.e., in all worlds if x is a non-mammal then we immediately know it's a non-cat (among other things). Someone with a different definition of 'cat,' or of 'GRT,' would have arrived at a different conclusion. But we can't just say willy-nilly that all truths are essentially true; otherwise the only possible world will be the actual world, perhaps a plausible claim metaphysically but not at all a plausible claim epistemically. (And real possibility is epistemic, not metaphysical.)

Also, 'GRT' is not in any case logically true; certainly it is not an axiom, and there is no reason to treat it as one.

Comment author: [deleted] 06 December 2012 08:05:06PM *  0 points [-]

I don't think we need this rule. It would make logical truths / tautologies meaningless, inexpressible, or magical. (We shouldn't dive into Wittgensteinian mysticism that readily.)

No, I didn't say that constraining possible worlds is a necessary condition on meaning. I said this:

1) A proposition is meaningful if and only if it is expressible by physics+logic, or merely by logic.

2) If a proposition is expressible by physics+logic, it constrains the possible worlds.

This leaves open the possibility of meaningful, non-world-constraining propositions (e.g. tautologies, such as the claims of logic), only they are not physics+logic expressible, but only logic expressible.

That depends on what you mean by "proposition." The written sentence "the cat is on the mat" could have been ungrammatical or semantically null, like "colorless green ideas sleep furiously."

That's not relevant to my point. I'd be happy to replace it with any proposition we can agree (for the sake of argument) to be meaningful. In fact, my argument will run with an unmeaningful proposition (if such a thing can be said to exist) as well.

On the other hand, if by "proposition" you mean "the specific meaning of a sentence,"

No, this isn't what I mean. By 'proposition' I mean a sentence, considered independently of its particular manifestation in a language. For example, 'Schnee ist weiss' and 'Snow is white' express the same proposition. Saying and writing 'shnee ist weiss' express the same proposition.

This looks like a problem of self-reference, but it's really a problem of essence-selection. When we identify something as 'the same thing' across multiple models or possible worlds...

I didn't understand this. Propositions (as opposed to things which express propositions) are not "in" worlds, and nothing of my argument involved identifying anything across multiple worlds. EY's OP stated that in order for an [empirical] claim to be meaningful, it has to constrain possible worlds, e.g. distinguish those worlds in which it is true from those in which it is false. Since a statement about the meaningfulness of propositions doesn't do this (i.e. it's a priori true or false of all possible worlds), it cannot be an empirical claim.

So I haven't said anything about essence, nor does any part of my argument require reference to essence.

Also, 'GRT' is not in any case logically true; certainly it is not an axiom, and there is no reason to treat it as one.

Agreed, it is not a merely logical claim. Given that it is also not an empirical (i.e. a physics+logic claim), and given my premise (1), which I take EY to hold, then we can conclude that the GRT is meaningless.

Comment author: Eugine_Nier 06 December 2012 05:38:14AM 9 points [-]

Only because you're cheating by reclassifying all cases where it was wrong as cases where we haven't figure out how to properly apply it yet.

Comment author: shminux 05 December 2012 06:05:08PM *  12 points [-]

Further to my other comment, how would one define a counterfactual in the Game of Life? Surely we should be able to analyze this simple case first if we want to talk about counterfactuals in the "real world"?

Comment author: faul_sname 06 December 2012 06:38:09PM *  6 points [-]

Say we have a blank grid. It would be reasonable to say "if this blank grid had a glider, the glider would move up and left" even if there is no actual glider on the grid. You can still make a mental model of what would happen in a changed grid, even if that grid isn't instantiated. I chose the example of a glider to show that you don't actually have to run a step-by-step simulation of the grid to predict behavior and thus emphasize that a counterfactual is a mental model, not an actual universe. Counterfactuals require a universe and a model that is isomorphic to that universe in some way, but the isomorphism doesn't have to be perfect.

Comment author: shminux 06 December 2012 07:25:46PM 1 point [-]

I like this example, and it counts as a counterfactual in our universe, where there is no actual glider drawn on an actual blank grid, but I am not sure it would count as a counterfactual in a GoL universe, unless you define such a universe to contain only a single blank canvas and nothing else.

Comment author: faul_sname 06 December 2012 07:32:35PM 2 points [-]

So what you're saying is that if we did define such a universe to contain only a single blank canvas and nothing else, our internal model of a grid with a glider would be a good example of a counterfactual?

(thus demonstrating that counterfactuals can, themselves, contain counterfactuals).

Comment author: shminux 06 December 2012 07:48:57PM 0 points [-]

(thus demonstrating that counterfactuals can, themselves, contain counterfactuals).

Nice one.

I am trying to nail the definition of a counterfactual in a GoL universe. Clearly, if you define this universe as a blank canvas, every game is a counterfactual. However, if the GoL universe is a collection of all possible games (hello, Tegmark!!), then there are no counterfactuals of the type you describe in it. However, what army1987 suggested would probably still count as a counterfactual: given a realization of a game and a certain position in it, find whether another realization, with an extra glider, converges to the same position. The counterfactualness there comes from privileging one game from the lot, not from mapping it to our universe.

Comment author: army1987 06 December 2012 11:35:19AM 4 points [-]

You go back to an earlier state of the grid, erase a glider, and resume the simulation from there.

Comment author: shminux 06 December 2012 06:11:45PM *  3 points [-]

erase a glider

A few thoughts on the matter.

What you suggest is one type of a counterfactual: change the state. Erasing a glider is, of course, illegal under the rules of the game, so to make it a legal game, you have to trace it backwards from the new state, or else you are not talking about the GoL anymore. This creates an interesting aside.

Like the real life, the Game of Life is not well-posed when run backwards: infinitely many configurations are legal just one simulation step back from a given one. This is because objects in the Game can die without a trace, and so can appear without a cause when run backward. This is similar to the way the world appears to us macroscopically: there is no way to tell the original shape of a drop of ink after it is dissolved in a bucket of water. This situation is known as the reversibility problem in cellular automata.

This freedom to create life out of nothing when simulating GoL backwards does not help us, however, in constructing the same starting configuration as the one with the glider not erased, because GoL is deterministic in the forward direction, and you cannot arrive at two different configurations when starting from the same one. But it does let us answer the following hypothetical: would adding a glider have made a difference in the future? I.e. would the glider in question collide with another object and disintegrate without a trace after several turns?

This "butterfly effect" investigation is trivial in the GoL and similar irreversible automata with simple rules, but it is quite suggestive if we consider the original question:

If Lee Harvey Oswald hadn't shot John F. Kennedy, someone else would've.

We can liken Oswald to your glider and see of removing it from the simulation ("counterfactual surgery") still results in the same final configuration (JFK shot). If so, we can declare the above statement to be "true", though not in the same sense as "Oswald shot JFK" is true, but in the same sense as a proved theorem is "true": its statement follows from its premises.

Comment author: IlyaShpitser 05 December 2012 12:55:38AM *  12 points [-]

From the logic point of view, counterfactuals are unproblematic, in that I can prove consistency of my favorite counterfactual logic by exhibiting a model. Then as far as a logician is concerned, we are done: our counterfactual worlds live in the mathematical structure of the exhibited model.


From the computer science point of view a little more is required, but as luck would have it, we can implement counterfactuals in some causal models. If your causal model is an actual circuit, then not only is it perfectly meaningful to ask "the output of the circuit is 1, what would be the output if I changed gate_0212 from OR to AND?" but it is possible to implement the counterfactual directly, and check. This is because we know enough about the causal model to ensure counterfactual invariance (e.g. other gates do not change). People use this kind of counterfactual reasoning to debug programs and circuits all the time! So from the "comp. sci" point of view, counterfactuals are unproblematic. The counterfactual universe "exists" in the operational sense of us having an effective procedure to get us there.


The problem arises when you are trying to deal with relatively poorly defined problems, like say problems in statistics or machine learning involving measurements of human populations or vitals in a patient with a ton of uncertainty about functional mechanisms and their invariance. Actually even in that case, people try to construct effective procedures to reach counterfactual universes, or something close (see, e.g. Imai's paper: http://imai.princeton.edu/research/Design.html). The question is then the following. Do counterfactual worlds in this case:

(a) not exist (ontological problem).

(b) exist, but we do not have a one to one mapping from the information we have to a unique counterfactual world describing the question we are interested in, even in principle (identification problem).

(c) exist, we do not have a one to one mapping from the information we have to a unique counterfactual world describing the question we are interested in, but we can get such a mapping if we learn a LOT more about the problem, and observe many many more variables (ignorance problem).

Comment author: Ritalin 07 December 2012 02:51:45PM 10 points [-]

I am finding the same problem with all articles in this sequence that I find with the explanation of Bayes' Theorem on Yudkowsky's main site. There are parts that seem so blindingly obvious they don't bear mentioning.

Yet soon thereafter, all of a sudden, I find myself completely lost. I can understand parts of the text separately, but can't link them together. I don't see where it comes from, where it's going, what problems it's addressing. I find it especially difficult to relate the illustrations to what's going on in the text.

I seldom have had this problem with the blog posts from the classical sequences (with some exceptions, such as his quantum physics sequence, which left me similarly confused).

Am I the only one who feels this way?

EDIT: upon reflection, this phenomenon, of feeling like there was a sudden, imperceptible jump from the boringly obvious to the utterly confusing, I've already experienced it before: in college, many lessons would follow this pattern, and it would take intensive study to figure out the steps the professor merrily jumped between what is, to them, two categories of the set of blindingly obvious things they already know and need to explain again. Maybe there's some sort of pattern there?

Comment author: Eliezer_Yudkowsky 08 December 2012 01:37:22AM 14 points [-]

This is a problem known as "bad writing" which I continue to struggle with, even after many years. Can you list the first part where you felt lost? Somewhere between there and the previous part, I must have skipped something.

I do hope people appreciate that all the "blindingly obvious" parts are parts where (at least in my guesstimation, and often in my actual experience) somebody else would otherwise get lost. The "obvious" is not the same for all people.

Comment author: Ritalin 08 December 2012 02:34:36AM *  3 points [-]

I would tell you about it, but now I'm afraid I'm distracting you from the latest chapter in Methods, which is kind of overdue and eagerly expected (and half of a Na No Wri Mo novel's wordcount? what exactly have you been up to?). I swear I'll take the time to go through the sequence and identify and point out the points at which I got lost, but first I'll wait for you to publish that chapter.

And yes, I know that one person's obvious is another's opaque; after all, that is the very root of this very problem.

@Donvoters: I am genuinely sorry; I'm just being honest here. This is like being addicted to a drug and, after months of waiting, hearing that the next batch is imminent and huge. I'm sort of fretting right now, and I'm probably not the only one.

Comment author: NancyLebovitz 28 December 2012 01:30:28PM 2 points [-]

Did you get back to Eliezer about what you found difficult in Mixed Reference?

Comment author: Ritalin 28 December 2012 03:33:53PM 0 points [-]

I had forgotten. Thanks for reminding me.

Comment author: lukeprog 08 December 2012 12:47:36PM 1 point [-]

I'll be linking to this comment pretty often, I think, to reply to commentors on my own posts.

Comment author: Ritalin 29 December 2012 08:10:22AM *  0 points [-]

So, first of all, I'm going to complain that doing this was a pain in the neck, and that commenting/editing would be much easier on Gdocs or on some similar application. In fact, I used Gdocs to write this, because doing so on the LW interface would have been intolerable. Still, there you are;

“A single discrete element of fundamental physics”

I suppose you mean an “elementary particle”? Took me a second to get it; it’s not the standard expression.

different low-level physical states are inside or outside the mental image of "some apples on the table" or alternatively "a kitten on the table"

I found this frankly misleading. When you say “mental image”, I think of an actual visualization, which is not a category a “low-level physical state” can belong to (or be “inisde of”). “Mental configuration” or “mental arrangement” might be more appropriate, and “corresponding” or “not corresponding” sound more acceptable. However, I’d rephrase the entire thing differently, as “different low-level physical states whose observation would result in a mental image of some apples on the table or a kitten on the table”.

The picture underneath is confusing because the previous paragraph makes us expect a “brain” or a “head” “visualizing” the “high states”, not the “high states” being somehow (one is function of the other, a correspondence? identification? belonging) linked to the “this actual universe in all its low-level glory” picture. I also find the choice of fuzziness around the edges of picture fragments, and the use of dotted lines, to be rather jarring. Is it supposed to be cute? Because what it conveys to me is “we’re not sure” and “the concept is unclear” and “the correspondence is distant or uncertain”, and that contrasts strongly with the actual text, which is much more rigorous. At the very least, you may want the line from “the Universe” to “all possible worlds” to end in a thicker dot, and to distort the shape of “all the possible worlds that would result in “a bunch of apples on the table” (that’s what the dotted circle means, right?) to be bigger and more potato-shaped or something, as is traditional to denote “abstract set of stuff whose shape doesn’t matter”; a circle seems too regular, and, in fact, I originally thought it represented a point, not a set. Its shape should also be different from the shape of the “we observe that a cat is on the table” set of possible universes, so as not to imply any relationship between the two.

  • but I'm not going to draw the image for that one. (We tried, and it came out too crowded.)

Did you need to mention that? Every time I read it, I get distracted wandering what it would have looked like. Perhaps it would be better to make the picture, and to hell with crowdedness.

Constraining this output constrains the possible states of the original, physical input universe:

On the picture next, I would have put the points of the arrows in the other direction, since that’s the direction of the causality link; universe-observation-model-calculation-six.

fulfilled by a mixture of physical reality and logical validity

“Mixture” sounds a little too anarchic, it confused me for a while. Doesn’t “physical reality” come before “logical validity”? What do you think of “composition” instead? It implies an order, that one is compounded over the other. “Combination”, which you used later, seems good too.

"running a logical function over the physical universe"

Sounds like an abuse of language. Wouldn’t lengthening it to "running a logical function over a model of the physical universe" or "running a logical function over an observation of the physical universe" be a good tradeoff?

(I haven't had time to go into this last part but it's an already-popular idea in philosophy of computation.)

I got distracted again.

And the Great Reductionist Thesis can be seen as the proposition that everything meaningful can be expressed this way eventually.

Is it true then, that “The GRT defines ‘meaningful’ as equivalent to ‘can be expressed this way’, and thus postulates that things that cannot be expressed this way are meaningless?” How do we avoid Wittgensteinonsense?

self-sensitization

? You mean becoming sensitive to one’s own state of mind? “I notice that I am confused”?

unless you believe the Illuminati planned it all

How about the more impartial (and factual, and logical) “unless you don’t believe LHO acted by himself”? It seems unfair to promote to attention, of all the vast field of hypotheses, a Bavarian organization that seems to have been ended circa 1787. You should avoid making jokes that will make many laugh at the expense of pissing off others; it’s kind of a terrible PR strategy.

For the record, I don’t “believe” in any specific conspiracy theory, and I assign highest probability to the “lone nutter” chain of events, but I assign the “not a lone nutter” set of hypotheses a probability that is significantly above zero; I don’t presume to promote to attention any particular hypothesis of that set with the evidence currently available to the public. If this position deserves mockery, I would like to know it. If it doesn’t, I would like people to stop acting as if the only options were “accept the standard version and only the standard version” or “choose one elaborate conspiracy theory and stick to it in the face of all evidence (or lack thereof)”.

For instance, about the moon landing; if you want to use a fact that is caused by Kennedy’s election and which wouldn’t have happened otherwise, how about “Monroe Cake” instead, which isn’t a potshot at anyone? And yes, I believe there was a moon landing, in the exact way the tale was officially told, until and unless I’m presented with sufficient evidence of the contrary, which hasn’t happened yet and which I don’t anticipate happening. I just don’t endorse antagonizing people, or otherwise raising tensions, unless you have to.

a nice neighborhood-structure

?

do not in fact actually exist.

I thought many-worlds implied they did exist “somewhere”?

And the same law could've just as easily have said that you're likely to find yourself in a world that goes over the integral of modulus to the power 1.99999?

“Finally, someone who speaks English!” Says Tony Stark and not many humans besides him.

I don’t have the equation in my head. Perhaps you could link to it?

a deterministic causal equation relating complex amplitudes inside a configuration space

Consider using parenthesis instead of a comma; I had to backtrack at the second semicolon, having thought that it was the second kind of stuff (and then remembering that, had the list items been separated by commas, you’d have used a colon and not a semicolon).

Why don’t you use bullet points and numbered lists more often? They’d make reading less fluid, but they’d also make some of your paragraphs much clearer, I think.

magical-reality-fluid

A bit of a distracting concept. How about the Pratchett formulation instead: thingness? It’s etymologically correct, and quite evocative.

This is just the same sort of problem if you say that causal models are meaningful and true relative to a mixture of three kinds of stuff, actual worlds, logical validities, and counterfactuals, and logical validities

Now this gets really confusing. Why are “logical validities” counted twice?

qualia are fundamental

I always thought ‘qualia’ was singular... Still, a link to Wikipedia would not be unwelcome; I’m having trouble parsing the sentence. “build references”? You seem to imply that they’re wrong for doing so, yet don’t seem to explicit why.

The whole paragraph on the Anthropic Trilemma has left me confuzzled. Then I clicked the link, saw the lengthy article, and thought “not today”. Maybe it would be beneficial to put a header/abstract/summary on top of your old sequences articles, for those of us who want to revise the old stuff but don’t want to have to read the whole thing all over again.

And -alas- the paragraph on modern philosophy ultimately leaves me with nothing other than “EY thinks modern philosophy is doing stuff that seems obviously stupid or half-baked”. Not the sort of thing you should do lightly; a link to something more developed would be good.

This is confusing the project of getting the gnomes out of the haunted mine, with trying to unmake the rainbow.

A couple links here would be nice.

And, yeah, what's a Born probability?

Comment author: BerryPick6 08 December 2012 09:19:22PM 0 points [-]

Can you list the first part where you felt lost?

When reading your work, I often share the feeling that Ritalin just described. In this particular instance, I was with you up until you started talking about the Born probabilities and then I just felt totally lost.

Comment author: gwern 07 December 2012 05:44:46PM 4 points [-]

Ah yes. Have you read about 'inferential distance' yet? :)

Comment author: Ritalin 07 December 2012 07:00:33PM 6 points [-]

Yes, I knew about them. I try to shorten them it in everything I do, from my vocabulary register to the concepts I use, which I try to make as rent-paying and empirical as possible. It's heavier work than I foresaw.

This has moved me from "impossible-to-understand nerd who talks down to you from an impenetrable ivory tower" to "that creepy guy who talks in punches and has strange ideas that make sense". Or, if you will, from a Sheldon Cooper to a coolness-impaired Tyler Durden. Socially, it wasn't a big gain.

Comment author: army1987 08 December 2012 12:27:22AM 2 points [-]

That's more or less how I felt about Penrose's The Road to Reality.

The great thing about talking with someone in person (or at least, in real-time one-to-one conversations) is that you can first assess how large the inferential distance is, e.g. “What are you working on?” “Cosmic rays. Do you know what cosmic rays are?” “No.” “Do you know what subatomic particles are?” “No.” “Do you know what an atom is?” “Yes.”

Comment author: Ritalin 08 December 2012 02:38:36AM *  2 points [-]

Cosmic rays. Do you know what cosmic rays are?” “No.”

You just have to hope they won't Wheatley they way around your questions and try to feign understanding things they don't, treating knowledge like a status game. That can really put a damper on meaningful communication.

Comment author: army1987 08 December 2012 10:36:53AM 1 point [-]

I don't think that ever happened to me -- at worst, they incorrectly believed that the understanding they had got from popularizations was accurate. But pretty much everybody at some point admits “I wish I could understand everything of that, but that sounds cool”, except people who actually understand (as evidenced by the fact that they ask questions too relevant for them to be just parroting stuff to hide ignorance).

(I guess the kind of people who treat everything like a status game would consider knowledge about sciency topics to be nerdy and therefore uncool.)

Comment author: Qiaochu_Yuan 08 December 2012 10:58:42AM *  4 points [-]

One way to treat knowledge like a status game is to be a "science fan." This is a game you play with other "science fans," and you win by knowing more "mind-blowing facts" about science than other people. It is popular on Quora.

Comment author: MrMind 07 December 2012 05:27:07PM 2 points [-]

Am I the only one who feels this way?

Absolutely not, it's quite a common feeling among mathematicians :)

Comment author: Ritalin 07 December 2012 06:44:47PM 3 points [-]

Ah, yes, the mathematician's double take. One should be wary of those, especially at a high level; when an elder mathematician wants to skip inferential steps for the sake of expediency, there's a chance that "then a miracle occurs" is somewhere in that mess of a blackboard.

In fact, the whole point of having a younger chevruta is so that they can point out that kind of details the bigger, more inferentially-distant minds might accidentally gloss over. They're like the great writer's spell-checker. Or like the comment section for Yudkowsky's blog posts.

Joking aside, I was actually wondering if others here felt the same way as I about EY's latest sequence of posts.

Comment author: CronoDAS 05 December 2012 03:10:23AM 9 points [-]

Incidentally, I'd give a probability of about 0.1 to the statement "If Lee Harvey Oswald hadn't shot John F. Kennedy, someone else would have" - there have been many people who have tried to assassinate Presidents.

Comment author: Nisan 05 December 2012 05:46:39AM *  11 points [-]

I was going to challenge you to a wager, but then I realized that (1) I agree with your estimate, and (2) I don't know how we'd settle a wager about a counterfactual.

Comment author: shminux 05 December 2012 06:00:43PM 1 point [-]

I don't know how we'd settle a wager about a counterfactual.

I guess this is my main issue with the whole sequence. No way to settle a wager means in my mind that there is no way to ascertain the truth of a statement, no matter how much physics, math and logic you throw at it.

EDIT: Trying to steel-man the game of counterfactuals: One way to settle the wager would be to run a simulation of the world as is, watch the assassination happen in every run, then do a tiny change which leads to no measurable large-scale effects (no-butterflies condition), except "Lee Harvey Oswald hadn't shot John F. Kennedy".

But what does "Lee Harvey Oswald hadn't shot John F. Kennedy" mean, exactly? He missed? Kennedy took a different route? Oswald grew up to be an upstanding citizen?

One can imagine a whole spectrum of possible counterfactual Kennedy-lives (KL) worlds, some of which are very similar to ours up to the day of the shooting, and others not so much. What properties of this spectrum would constitute a winning wager? Would you go for "every KL world has to be otherwise indistinguishable (by what criteria? Media headlines?) from ours"? Or "there is at least one KL world like that"? Or something in between? Or something totally different?

Until one drill down and settles the definition of a counterfactual, probably in a way similar to the above, I see no way to meaningfully discuss the issue.

Comment author: Eliezer_Yudkowsky 05 December 2012 09:25:51PM 9 points [-]

I don't know how we'd settle a wager about a counterfactual.

I guess this is my main issue with the whole sequence.

That's the point of this post. Only causal models can be settled. Counterfactuals cannot be observed, and can only be derived as logical constructs via axiomatic specification from the causal models which can be observed.

Comment author: faul_sname 05 December 2012 06:30:14PM 3 points [-]

You don't have to construct the model at that level of detail to meaningfully discuss the issue. Just look at the base rate of presidential assassinations and update that to cover the large differences with the Kennedy case. If you're trying to simulate a universe without Lee Harvey Oswald, you're probably overfitting, particularly if you're a human. Your internal model of how Kennedy was actually shot doesn't contain a high-fidelity of the world in which Oswald grew up and went through a series of mental states that culminated with him shooting Kennedy (or at least, you're not simulating each mental state to come to the outcome). Instead, you have a model of the world in which Lee Harvey Oswald shoots JFK, and otherwise doesn't really factor into your model. While removing Oswald from the real world would have large effects, removing him from your model doesn't.

I think that you ask "what are the chances that Kennedy would have been shot if Oswald hadn't done it?" you're probably asking something along the lines of "If I build the best model I can of the world surrounding that event, and remove Oswald, does the model show Kennedy getting shot, and if so, with what confidence?" So in order to settle the wager, you would have to construct a model of the world that both of you agreed made good enough predictions (probably by giving it information about the state of society at various times and seeing how often it predicts a presidential assassination) and seeing what the answer it spits out is. There might be a problem of insufficient data, but it seems pretty clear to me that when we talk about counterfactuals, we're talking about models of the world that we alter, not actual, existing worlds. If many worlds was false and there was only one, fully deterministic universe (that contained humans), we would still talk about counterfactuals. Unless I'm missing something obvious.

Comment author: fubarobfusco 05 December 2012 09:47:22PM 1 point [-]

Your internal model of how Kennedy was actually shot doesn't contain a high-fidelity of the world in which Oswald grew up and went through a series of mental states that culminated with him shooting Kennedy

Well, my model has Oswald in the Marines with Kerry Thornley — aka Lord Omar, of Discordian legend — and a counterfactual in which a slightly more tripped-out conversation between the two would have led to Oswald becoming an anarchist instead of a Marxist; thus preventing his defection to the Soviet Union ....

Comment author: TraderJoe 05 December 2012 09:37:41AM 1 point [-]

And many people who have tried to assassinate Kennedys...

Comment author: Jayson_Virissimo 05 December 2012 08:08:40AM 8 points [-]

Take the universe and grind it down to the finest powder and sieve it through the finest sieve and then show me one atom of justice, one molecule of mercy.

Take the universe and grind it down to the finest powder and sieve it through the finest sieve and then show me one atom of time, one molecule of velocity. Oh wait...

Comment author: cousin_it 05 December 2012 09:47:59AM *  10 points [-]

Yeah, with "atoms of threeness" Eliezer seems to have narrowly missed an interesting point. Multiplying apples to get square apples makes no sense, but if we'd divided them instead, we'd notice that the universe contains dimensionless constants - if the universe can be said to "contain" anything at all, like atoms or velocity.

Comment author: Vaniver 05 December 2012 03:19:51AM 7 points [-]

three kinds of stuff, actual worlds, logical validities, and counterfactuals, and logical validities.

This list contains duplicate elements.

Comment author: dspeyer 05 December 2012 04:44:22AM 6 points [-]

I realize this is a small thing, but this essay appears to use "fact" to mean "a statement sufficiently well-formed to be either true or false" rather than "a statement which is true" and that kept distracting me from its actual point. Can some other word be found?

Comment author: Tyrrell_McAllister 06 December 2012 05:38:11PM 0 points [-]

Has the post been edited since you made this comment? I couldn't find any examples of this.

Comment author: dspeyer 07 December 2012 06:17:11AM 0 points [-]

"If Oswald hadn't shot Kennedy, nobody else would've" is a fact

Comment author: Tyrrell_McAllister 07 December 2012 12:18:43PM 0 points [-]

He is saying that that is a fact, but not merely because it is "a statement sufficiently well-formed to be either true or false". For example, he would say that "If Oswald hadn't shot Kennedy, somebody else would've" is not a fact, even though it is equally well formed. The point of the article is to explain how some counterfactuals can be facts while others are not.

Comment author: Eliezer_Yudkowsky 05 December 2012 12:22:10AM 4 points [-]

Mainstream status:

AFAIK, the proposition that "Logical and physical reference together comprise the meaning of any meaningful statement" is original-as-a-whole (with many component pieces precedented hither and yon). Likewise I haven't elsewhere seen the suggestion that the great reductionist project is to be seen in terms of analyzing everything into physics+logic.

An important related idea I haven't gone into here is the idea that the physical and logical references should be effective or formal, which has been in the job description since, if I recall correctly, the late nineteenth century or so, when mathematics was being axiomatized formally for the first time. This pat is popular, possibly majoritarian; I think I'd call it mainstream. See e.g. http://plato.stanford.edu/entries/church-turing/ although logical specifiability is more general than computability (this is also already-known).

Obviously and unfortunately, the idea that you are not supposed to end up with more and more ontologically fundamental stuff is not well-enforced in mainstream philosophy.

Comment author: Alejandro1 05 December 2012 02:13:01AM 18 points [-]

AFAIK, the proposition that "Logical and physical reference together comprise the meaning of any meaningful statement" is original-as-a-whole (with many component pieces precedented hither and yon). Likewise I haven't elsewhere seen the suggestion that the great reductionist project is to be seen in terms of analyzing everything into physics+logic.

This seems awfully similar to Hume's fork:

If we take in our hand any volume; of divinity or school metaphysics, for instance; let us ask, Does it contain any abstract reasoning concerning quantity or number? No. Does it contain any experimental reasoning concerning matter of fact and existence? No. Commit it then to the flames: for it can contain nothing but sophistry and illusion.

  • David Hume, An Enquiry Concerning Human Understanding (1748)

As Mardonius says, 20th century logical empiricism (also called logical positivism or neopositivism) is basically the same idea with "abstract reasoning" fleshed out as "tautologies in formal systems" and "experimental reasoning" fleshed out initially as " statements about sensory experiences". So the neopositivists' original plan was to analyze everything, including physics, in terms of logic + sense data (similar to qualia, in modern terminology). But some of them, like Neurath, considered logic + physics a more suitable foundation from the beginning, and others, like Carnap, became eventually convinced of this as well, so the mature neopositivist position is quite similar to yours.

One key difference is that for you (I think, correct me if I am wrong) reductionism is an ontological enterprise, showing that the only "stuff" there is (in some vague sense) is logic and physics. For the neopositivists, such a statement would be as meaningless as the metaphysics they were trying to "commit to the flames". Reductionism was a linguistic enterprise: to develop a language in which every meaningful statement is translatable into sentences about physics (or qualia) and logic, in order to make the sciences more unified and coherent and to do away with muddled metaphysical thought.

Comment author: Eliezer_Yudkowsky 05 December 2012 02:31:14AM 4 points [-]

Is there a good statement of the "mature neopositivist" / Carnap's position?

Comment author: Alejandro1 05 December 2012 10:48:42PM *  2 points [-]

There is no article on Carnap on the SEP, and I couldn't find a clear statement on the Vienna Circle article, but there is a fairly good one in the Neurath article:

In his classic work Der Logische Afbau der Welt (1928) (known as the Aufbau and translated as The Logical Structure of the World), Carnap investigated the logical ‘construction’ of objects of inter-subjective knowledge out of the simplest starting point or basic types of fundamental entities (Russell had urged in his late solution to the problem of the external world to substitute logical constructions for inferred entities). He introduced several possible domains of objects, one of which being the psychological objects of private sense experience—analysed as ‘elementary experiences’.

(…)

Neurath first confronted Carnap on yet another alleged feature of his system, namely, subjectivism. He promptly rejected Carnap's proposals on the grounds that if the language and the system of statements that constitute scientific knowledge are intersubjective, then phenomenalist talk of immediate subjective, private experiences should have no place.

(…)

Following Neurath, Carnap explicitly opposed to the language of experience a narrower conception of intersubjective physicalist language which was to be found in the exact quantitative determination of physics-language realized in the readings of measurement instruments. Remember that for Carnap only the structural or formal features, in this case, of exact mathematical relations (manifested in the topological and metric characteristics of scales), can guarantee objectivity. After the Aufbau, now the unity of science rested on the universal possibility of the translation of any scientific statement into physical language—which in the long run might lead to the reduction of all scientific knowledge to the laws and concepts of physics.

The mature Carnap position seems to be, then, not to reduce everything to logic + fundamental physics (electrons/wavefunctions/etc), as perhaps you thought I had implied, but to reduce everything to logic + observational physics (statements like "Voltimeter reading = 10 volts"). Theoretical sentences about electrons and such are to be reduced (in some sense that varied which different formulations) to sentences of observational physics. This does not mean that for Carnap electrons are not "real"; as I said before, reductionism was conceived as a linguistic proposal, not an ontological thesis.

Comment author: Eliezer_Yudkowsky 05 December 2012 11:23:00PM 0 points [-]

Experience + logic != physics + logic > causality + logic

Comment author: shminux 05 December 2012 11:26:56PM -1 points [-]

Experience + models = reality

Comment author: RobbBB 06 December 2012 05:25:40AM *  2 points [-]

Cucumbers are neither experiences nor models. Yet I'm pretty sure reality includes at least one cucumber.

Comment author: shminux 06 December 2012 06:30:45AM *  1 point [-]

Cucumbers are both experiences and models, actually. You experience its sight, texture and taste, you model this as a green vegetable with certain properties which predict and constrain your similar future experiences.

Numbers, by comparison, are pure models. That's why people are often confused about whether they "exist" or not.

Comment author: [deleted] 06 December 2012 08:21:33PM 0 points [-]

You experience its sight, texture and taste, you model this as a green vegetable with certain properties which predict and constrain your similar future experiences.

Are experiences themselves models? If not, are you endorsing the view that qualia are fundamental?

Comment author: shminux 06 December 2012 09:19:12PM 1 point [-]

Are experiences themselves models? If not, are you endorsing the view that qualia are fundamental?

Experiences are, of course, themselves a multi-layer combination of models and inputs, and at some point you have to stop, but qualia seem to be at too high a level, given that they appear to be reducible to physiology in most brain models.

Comment author: RobbBB 06 December 2012 08:33:51AM 0 points [-]
  1. How do you know models exist, and aren't just experiences of a certain sort?

  2. How do you know that unexperienced, unmodeled cucumbers don't exist? How do you know there was no physical universe prior to the existence of experiencers and modelers?

Comment author: NancyLebovitz 07 December 2012 03:16:55PM *  3 points [-]

I've played with the idea that there is nothing but experience (Zen and the Art of Motorcycle Maintenance was rather convincing). However, it then becomes surprising that my experience generally behaves as though I'm living in a stable universe with such things as previously unexperienced cucumbers showing up at plausible times.

Comment author: bryjnar 05 December 2012 11:11:50PM 1 point [-]

Even just take the old logical postivist doctrine about analyticity/syntheticity: all statements are either "analytic" (i.e. true by logic (near enough)), or synthetic (true due to experience). That's at least on the same track. And I'm pretty sure they wouldn't have had a problem with statements that were partially both.

Comment author: crazy88 05 December 2012 12:45:55AM 8 points [-]

Obviously and unfortunately, the idea that you are not supposed to end up with more and more ontologically fundamental stuff inside your philosophy is not mainstream.

I think I must be misunderstanding what you're saying here because something very similar to this is probably the principle accusation relied upon in metaphysical debates (if not the very top, certainly top 3). So let me outline what is standard in metaphysical discussions so that I can get clear on whether you're meaning something different.

In metaphysics, people distinguish between quantitative and qualitative parsimony. Quantitative parisimony is about the amount of stuff your theory is committed to (so a theory according to which more planets exist is less quantitatively parsimonious than an alternative). Most metaphysicians don't care about quantative parsimony. On the other hand, qualitative parsimony is about the types of stuff that your theory is committed to. So if a theory is committed to causation and time, this would be less qualitatively parsimonious than one that that was only committed to causation (just an example, not meant to be an actual case). Qualitative parsimony is seen to be one of the key features of a desirable metaphysical theory. Accusations that your theory postulates extra ontological stuff but doesn't gain further explanatory power for doing so is basically the go to standard accusation against a metaphysical theory.

Fundamentality is also a major philosophical issue - the idea that some stuff you postulate is ontologically fundamental and some isn't. Fundamentality views are normally coupled with the view that what really matters is qualitative parsimony of fundamental stuff (rather than stuff generally).

So how does this differ from the claim that you're saying is not mainstream?

Comment author: Eliezer_Yudkowsky 05 December 2012 12:53:53AM 2 points [-]

The claim might just need correction to say, "Many philosophers say that simplicity is a good thing but the requirement is not enforced very well by philosophy journals" or something like that. I think I believe you, but do you have an example citation anyway? (SEP entries or other ungated papers are in general good; I'm looking for an example of an idea being criticized due to lack of metaphysical parsimony.) In particular, can we find e.g. anyone criticizing modal logic because possibility shouldn't be basic because metaphysical parsimony?

Comment author: crazy88 05 December 2012 01:11:26AM *  9 points [-]

In terms of Lewis, I don't know of someone criticising him for this off-hand but it's worth noting that Lewis himself (in his book On the Plurality of Worlds) recognises the parsimony objection and feels the need to defend himself against it. In other words, even those who introduce unparsimonious theories in philosophy are expected to at least defend the fact that they do so (of course, many people may fail to meet these standards but the expectation is there and theories regularly get dismissed and ignored if they don't give a good accounting of why we should accept their unparsimonious nature).

Sensations and brain processes: one of Jack Smart's main grounds for accepting the identity theory of mind is based around considerations of parsimony

Quine's paper On What There Is is basically an attack on views that hold that we need to accept the existence of things like pegasus (because otherwise what are we talking about when we say "Pegasus doesn't exist"). Perhaps a ridiculous debate but it's worth noting that one of Quine's main motivations is that this view is extremely unparsimonious.

From memory, some proponents of EDT support this theory because they think that we can achieve the same results as CDT (which they think is right) in a more parsimonious way by doing so (no link for that however as that's just vague recollection).

I'm not actually a metaphysician so I can't give an entire roll call of examples but I'd say that the parsimony objection is the most common one I hear when I talk to metaphysicians.

Comment author: Eugine_Nier 06 December 2012 06:22:31AM 2 points [-]

In particular, can we find e.g. anyone criticizing modal logic because possibility shouldn't be basic because metaphysical parsimony?

Why shouldn't it? I haven't seen any reduction of it that deals with this objection.

Comment author: Peterdjones 05 December 2012 02:06:33PM *  0 points [-]

"Many philosophers say that simplicity is a good thing but the requirement is not enforced very well by philosophy journals"

Would that be desirable? If a contributor can argue persuasively for dropping parsimony, why should that be suppressed?

criticizing modal logic because possibility

Surely that should be modal realism.

Comment author: NancyLebovitz 05 December 2012 03:32:27PM 1 point [-]

"Make things as simple as possible, but no simpler." --Albert Einstein

How do you know whether something is as simple as possible?

In terms of publishing, should the standard be as simple as is absolutely possible, or should it be as simple as possible given time and mental constraints?

Comment author: DanArmak 05 December 2012 08:10:15PM 0 points [-]

How do you know whether something is as simple as possible?

You keep trying to make it simpler, but you fail to do so without losing something in return.

Comment author: crazy88 05 December 2012 09:49:04PM *  2 points [-]

It still may be hard to resolve when something is as simple as possible.

So modal realism (the idea that possible worlds exist concretely) has been highlighted a few times in this thread as an unparsimonious theory but Lewis has two responses to this:

1.) This is (at least mostly) quantitative unparsimony not qualitative (lots of stuff, not lots of types of stuff). It's unclear how bad quantitative unparsimony is. Specifically, Lewis argues that there is no difference between possible worlds and actual worlds (actuality is indexical) so he argues that he doesn't postulate two types of stuff (actuality and possibility) he just postulates a lot more of the stuff that we're already committed to. Of course, he may be committed to unicorns as well as goats (which the non-realist isn't) but then you can ask whether he's really committed to more fundamental stuff than we are.

2.) Lewis argues that his theory can explain things that no-one else can so even if his theory is less parsimonious, it gives rewards in return for that cost.

Now many people will argue that Lewis is wrong, perhaps on both counts but the point is that even with the case that's been used almost as a benchmark for unparsimonious philosophy in this thread, it's not as simple as "Lewis postulates two types of stuff when he doesn't need to, therefore, clearly his theory is not as simple as possible."

Comment author: Mardonius 05 December 2012 12:55:10AM 3 points [-]

Isn't this, essentially, a mild departure from late Logical Empiricism to allow for a wider definition of Physical and a more specific definition of Logical references?

Comment author: Eliezer_Yudkowsky 05 December 2012 12:59:26AM 1 point [-]

I don't see anything similar to this post on a quick skim of http://plato.stanford.edu/entries/logical-empiricism/ . Please specify.

Comment author: Mardonius 05 December 2012 01:55:25AM 2 points [-]

Well, I was specifically thinking of this passage

The Great Reductionist Project can be seen as figuring out how to express meaningful sentences in terms of a >combination of physical references (statements whose truth-value is determined by a truth-condition directly >correspnding to the real universe we're embedded in) and logical references (valid implications of premises, >or elements of models pinned down by axioms); where both physical references and logical references are to >be described 'effectively' or 'formally', in computable or logical form. (I haven't had time to go into this last part >but it's an already-popular idea in philosophy of computation.)

And the Great Reductionist Thesis can be seen as the proposition that everything meaningful can be >expressed this way eventually.

Which, to my admittedly rusty knowledge of mid 20th century philosophy, sounds extremely similar to the anti-metaphysics position of Carnap circa 1950. His work on Ramsey sentences, if I recall, was an attempt to reduce mixed statements including theoretical concepts ("appleness") to a statement consisting purely of Logical and Observational Terms. I'm fairly sure I saw something very similar to your writings in his late work regarding Modal Logic, but I'm clearly going to have to dig up the specific passage.

Comment author: RobbBB 05 December 2012 02:38:27AM *  8 points [-]

Amusingly, this endeavor also sounds like your arch-nemesis David Chalmers' new project, Constructing the World. Some of his moderate responses to various philosophical puzzles may actually be quite useful to you in dismissing sundry skeptical objections to the reductive project; from what I've seen, his dualism isn't indispensable to the interesting parts of the work.

Comment author: bryjnar 05 December 2012 03:47:37AM 3 points [-]

Just to say that in general, apart from the stuff about consciousness, which I disagree with but think is interesting, I think that Chalmers is one of the best philosophers alive today. Seriously, he does a lot of good work.

Comment author: Will_Newsome 06 December 2012 05:34:28PM *  4 points [-]

He also reads LessWrong, I think.

Comment author: Alejandro1 06 December 2012 05:56:57PM 4 points [-]

I am about 90% certain that he is djc.

Comment author: gwern 06 December 2012 06:00:06PM 4 points [-]

I'd agree; the link to philpapers (a Chalmers project), claiming to be a pro, having access to leading decision theorists - all consistent.

Comment author: RobbBB 06 December 2012 06:31:44PM 4 points [-]

It's either Chalmers or a deliberate impersonator. 'DJC' stands for 'David John Chalmers.'

Comment author: aaronsw 25 December 2012 08:57:42PM *  1 point [-]

It's too bad EY is deeply ideologically committed to a different position on AI, because otherwise his philosophy seems to very closely parallel John Searle's. Searle is clearer on some points and EY is clearer on others, but other than the AI stuff they take a very similar approach.

EDIT: To be clear, John Searle has written a lot, lot more than the one paper on the Chinese Room, most of it having nothing to do with AI.

Comment author: Eliezer_Yudkowsky 25 December 2012 09:14:15PM 0 points [-]

So... admittedly my main acquaintance with Searle is the Chinese Room argument that brains have 'special causal powers', which made me not particularly interested in investigating him any further. But the Chinese Room argument makes Searle seem like an obvious non-reductionist with respect to not only consciousness but even meaning; he denies that an account of meaning can be given in terms of the formal/effective properties of a reasoner. I've been rendering constructive accounts of how to build meaningful thoughts out of "merely" effective constituents! What part of Searle is supposed to be parallel to that?

Comment author: aaronsw 25 December 2012 09:27:36PM *  4 points [-]

I guess I must have misunderstood something somewhere along the way, since I don't see where in this sequence you provide "constructive accounts of how to build meaningful thoughts out of 'merely' effective constituents" . Indeed, you explicitly say "For a statement to be ... true or alternatively false, it must talk about stuff you can find in relation to yourself by tracing out causal links." This strikes me as parallel to Searle's view that consciousness imposes meaning.

But, more generally, Searle says his life's work is to explain how things like "money" and "human rights" can exist in "a world consisting entirely of physical particles in fields of force"; this strikes me as akin to your Great Reductionist Project.

Comment author: pjeby 26 December 2012 12:28:26AM 5 points [-]

Searle says his life's work is to explain how things like "money" and "human rights" can exist in "a world consisting entirely of physical particles in fields of force";

Someone should tell him this has already been done: dissolving that kind of confusion is literally part of LessWrong 101, i.e. the Mind Projection Fallacy. Money and human rights and so forth are properties of minds modeling particles, not properties of the particles themselves.

That this is still his (or any other philosopher's) life's work is kind of sad, actually.

Comment author: aaronsw 04 January 2013 10:01:47PM 2 points [-]

I guess my phrasing was unclear. What Searle is trying to do is generate reductions for things like "money" and "human rights"; I think EY is trying to do something similar and it takes him more than just one article on the Mind Projection Fallacy. (Even once you establish that it's properties of minds, not particles, there's still a lot of work left to do.)

Comment author: Peterdjones 27 December 2012 10:57:31AM -3 points [-]

Or maybe Searle is tackling a much harder version of the problem, for instance explaining how things like human rights and ethics can be binding or obligatory on people when they are "all in the mind", explaining why one person should be beholden to another's mind projection.

Comment author: Eliezer_Yudkowsky 26 December 2012 12:07:54AM 1 point [-]

This strikes me as parallel to Searle's view that consciousness imposes meaning.

Why? Did I mention consciousness somewhere? Is there some reason a non-conscious software program hooked up to a sensor, couldn't do the same thing?

I don't think Searle and I agree on what constitutes a physical particle. For example, he thinks 'physical' particles are allowed to have special causal powers apart from their merely formal properties which cause their sentences to be meaningful. So far as I'm concerned, when you tell me about the structure of something's effects on the particle fields, there shouldn't be anything left after that - anything left is extraphysical.

Comment author: Peterdjones 27 December 2012 11:05:15AM 1 point [-]

Searle's views have nothing to do with attributing novel properties to fundamental particles. They are more to do with identifying mental properties with higher-levle physical properties, which are themselves irreducible in a sense (but also reducible in another sense).

Comment author: pjeby 25 December 2012 09:12:04PM -1 points [-]

It's too bad EY is deeply ideologically committed to a different position on AI, because otherwise his philosophy seems to very closely parallel John Searle's

Perhaps I'm confused, but isn't Searle the guy who came up with that stupid Chinese Room thing? I don't see at all how that's remotely parallel to LW philosophy, or why it would be a bad thing to be ideologically opposed to his approach to AI. (He seems to think it's impossible to have AI, after all, and argues from the bottom line for that position.)

Comment author: aaronsw 25 December 2012 09:46:41PM 3 points [-]

I was talking about Searle's non-AI work, but since you brought it up, Searle's view is:

  1. qualia exists (because: we experience it)
  2. the brain causes qualia (because: if you cut off any other part of someone they still seem to have qualia)
  3. if you simulate a brain with a Turing machine, it won't have qualia (because: qualia is clearly a basic fact of physics and there's no way just using physics to tell whether something is a Turing-machine-simulating-a-brain or not)

Which part does LW disagree with and why?

Comment author: Benito 26 December 2012 01:01:07AM *  2 points [-]

To offer my own reasons for disagreement,

I think the first point is unfounded (or misguided). We do things (like moving, and thinking). We notice and can report that we've done things, and occasionally we notice and can report that we've noticed that we've done something. That we can report how things appear to a part of us that can reflect upon stimuli is not important enough to be called 'quaila'. That we notice that we find experience 'ineffable' is not a surprise either - you would not expect the brain to be able to report everything that occurs, down to the neurons firing (or atoms moving). So, all we really have is the ability to notice and report that which has been advantageous for us to report in the evolutionary history of the human (these stimuli that we can notice are called 'experiences'). There is nothing mysterious here, and the word 'qualia' always seems to be used mysteriously - so I don't think the first point carries the weight it might appear to.

if you simulate a brain with a Turing machine, it won't have qualia (because: qualia is clearly a basic fact of physics and there's no way just using physics to tell whether something is a Turing-machine-simulating-a-brain or not)

Qualia is not clearly a basic fact of physics. I made the point that we would not expect a species designed by natural selection to be able to report or comprehend its most detailed, inner workings, solely on the evidence of what it can report and notice. But this is all skirting around the core idea of LessWrong: The map is not the territory. Just because something seems fundamental does not mean it is. Just because it seems like a Turing machine couldn't be doing consciousness, doesn't mean that is how it is. We need to understand how it came to be that we feel what we feel, before go making big claims about the fundamental nature of reality. This is what is worked on in LessWrong, not in Searle's philosophy.

Comment author: nshepperd 26 December 2012 12:33:27AM *  2 points [-]

I can't really speak for LW as a whole, but I'd guess that among the people here who don't believe¹ "qualia doesn't exist", 1 and 2 are fine, but we have issues with 3, as expanded below. Relatedly, there seems be some confusion between the "boring AI" proposition, that you can make computers do reasoning, and Searle's "strong AI" thing he's trying to refute, which says that AIs running on computers would have both consciousness and some magical "intentionality". "Strong AI" shouldn't actually concern us, except in talking about EMs or trying to make our FAI non-conscious.

3. if you simulate a brain with a Turing machine, it won't have qualia

Pretty much disagree.

qualia is clearly a basic fact of physics

Really disagree.

and there's no way just using physics to tell whether something is a Turing-machine-simulating-a-brain or not

And this seems really unlikely.

¹ I qualify my statement like this because there is a long-standing confusion over the use of the word "qualia" as described in my parenthetical here.

Comment author: aaronsw 04 January 2013 10:08:32PM 2 points [-]

Well, let's be clear: the argument I laid out is trying to refute the claim that "I can create a human-level consciousness with a Turing machine". It doesn't mean you couldn't create an AI using something other than a pure Turing machine and it doesn't mean Turing machines can't do other smart computations. But it does mean that uploading a brain into a Von Neumann machine isn't going to keep you alive.

So if you disagree that qualia is a basic fact of physics, what do you think it reduces to? Is there anything else that has a first-person ontology the way qualia does?

And if you think physics can tell whether something is a Turing-machine-simulating-a-brain, what's the physical algorithm for looking at a series of physical particles and deciding whether it's executing a particular computation or not?

Comment author: nshepperd 05 January 2013 02:45:29AM 1 point [-]

So if you disagree that qualia is a basic fact of physics, what do you think it reduces to?

Something brains do, obviously. One way or another.

And if you think physics can tell whether something is a Turing-machine-simulating-a-brain, what's the physical algorithm for looking at a series of physical particles and deciding whether it's executing a particular computation or not?

I should perhaps be asking what evidence Searle has for thinking he knows things like what qualia is, or what a computation is. My statements were both negative: it is not clear that qualia is a basic fact of physics; it is not obvious that you can't describe computation in physical terms. Searle just makes these assumptions.

If you must have an answer, how about this: a physical system P is a computation of a value V if adding as premises the initial and final states of P and a transition function describing the physics of P shortens a formal proof that V = whatever.

Comment author: aaronsw 05 January 2013 09:46:46PM 0 points [-]

They're not assumptions, they're the answers to questions that have the highest probability going for them given the evidence.

Comment author: MugaSofer 26 December 2012 02:04:35AM 1 point [-]

if you simulate a brain with a Turing machine, it won't have qualia (because: qualia is clearly a basic fact of physics and there's no way just using physics to tell whether something is a Turing-machine-simulating-a-brain or not)

There's your problem. Why the hell should we assume that "qualia is clearly a basic fact of physics "?

Comment author: aaronsw 04 January 2013 10:10:16PM 1 point [-]

Because it's the only thing in the universe we've found with a first-person ontology. How else do you explain it?

Comment author: TheOtherDave 26 December 2012 12:57:03AM *  1 point [-]

Another not-speaking-for-LW answer:

Re #1: I certainly agree that we experience things, and that therefore the causes of our experience exist. I don't really care what name we attach to those causes... what matters is the thing and how it relates to other things, not the label. That said, in general I think the label "qualia" causes more trouble due to conceptual baggage than it resolves, much like the label "soul".

Re #2: This argument is oversimplistic, but I find the conclusion likely.
More precisely: there are things outside my brain (like, say, my adrenal glands or my testicles) that alter certain aspects of my experience when removed, so it's possible that the causes of those aspects reside outside my brain. That said, I don't find it likely; I'm inclined to agree that the causes of my experience reside in my brain. I still don't care much what label we attach to those causes, and I still think the label "qualia" causes more confusion due to conceptual baggage than it resolves.

Re #3: I see no reason at all to believe this. The causes of experience are no more "clearly a basic fact of physics" than the causes of gravity; all that makes them seem "clearly basic" to some people is the fact that we don't understand them in adequate detail yet.

Comment author: pjeby 26 December 2012 12:20:35AM 0 points [-]

Searle's view is:

  1. qualia exists (because: we experience it)
  2. the brain causes qualia (because: if you cut off any other part of someone they still seem to have qualia)
  3. if you simulate a brain with a Turing machine, it won't have qualia (because: qualia is clearly a basic fact of physics and there's no way just using physics to tell whether something is a Turing-machine-simulating-a-brain or not)

Which part does LW disagree with and why?

The whole thing: it's the Chinese Room all over again, a intuition pump that begs the very question it's purportedly answering. (Beginning an argument for the existence of qualia with a bare assertion that they exist is a little more obvious than the way that the word "understanding" is fudged in the Chinese Room argument, but basically it's the same.)

I suppose you could say that there's a grudging partial agreement with your point number two: that "the brain causes qualia". The rest of what you listed, however, is drivel, as is easy to see if you substitute some other term besides "qualia", e.g.:

  1. Free will exists (because: we experience it)
  2. The brain causes free will (because if you cut off any part, etc.)
  3. If you simulate a brain with a Turing machine, it won't have free will because clearly it's a basic fact of physics and there's no way to tell just using physics whether something is a machine simulating a brain or not.

It doesn't matter what term you plug into this in place of "qualia" or "free will", it could be "love" or "charity" or "interest in death metal", and it's still not saying anything more profound than, "I don't think machines are as good as real people, so there!"

Or more precisely: "When I think of people with X it makes me feel something special that I don't feel when I think of machines with X, therefore there must be some special quality that separates people from machines, making machine X 'just a simulation'." This is the root of all these Searle-ian arguments, and they are trivially dissolved by understanding that the special feeling people get when they think of X is also a property of how brains work.

Specifically, the thing that drives these arguments is our inbuilt machinery that classifies things as mind-having or not-mind-having, for purposes of prediction-making. But the feeling that we get that a thing is mind-having or not-mind-having is based on what was useful evolutionarily, not on what the actual truth is. Searlian (Surly?) arguments are thus in exactly the same camp as any other faith-based argument: elevating one's feelings to Truth, irrespective of the evidence against them.

Comment author: [deleted] 26 December 2012 01:07:45AM 1 point [-]

(Beginning an argument for the existence of qualia with a bare assertion that they exist is a little more obvious than the way that the word "understanding" is fudged in the Chinese Room argument, but basically it's the same.)

Just a nit pick: the argument Aaron presented wasn't an argument for the existence of qualia, and so taking the existence of qualia as a premise doesn't beg the question. Aaron's argument was an argument agains artificial consciousness.

Also, I think Aaron's presentation of (3) was a bit unclear, but it's not so bad a premise as you think. (3) says that since qualia are not reducible to purely physical descriptions, and since a brain-simulating turing-machine is entirely reducible to purely physical descriptions, brain-simulating turing-machines won't experience qualia. So if we have qualia, and count as conscious in virtue of having qualia (1), then brain-simulating turing machines won't count as conscious. If we don't have qualia, i.e. if all our mental states are reducible to purely physical descriptions, then the argument is unsound because premise (1) is false.

You're right that you can plug many a term in to replace 'qualia', so long as those things are not reducible to purely physical descriptions. So you couldn't plug in, say, heart-attacks.

This is the root of all these Searle-ian arguments, and they are trivially dissolved by understanding that the special feeling people get when they think of X is also a property of how brains work.

Could you explain this a bit more? I don't see how it's relevant to the argument. Searle is not arguing on the basis of any special feelings. This seems like a straw man to me, at the moment, but I may not be appreciating the flaws in Searle's argument.

Comment author: amcknight 07 December 2012 07:01:14AM 3 points [-]

where both physical references and logical references are to be described 'effectively' or 'formally', in computable or logical form.

Can anyone say a bit more about why physical references would need to be described 'effectively'/computably? Is this based on the assumption that the physical universe must be computable?

Comment author: MrMind 07 December 2012 05:25:57PM 0 points [-]

Can anyone say a bit more about why physical references would need to be described 'effectively'/computably?

I think because if they are described by an uncomputable procedure, one for example involving oracles or infinite resources, then they (with very high probability) would not be able to be computed by our brains.

Comment author: Eugine_Nier 08 December 2012 04:21:16AM 2 points [-]

So? Use said oracles to upgrade our brains.

Comment author: amcknight 08 December 2012 09:16:10PM 0 points [-]

This crossed my mind, but I thought there might be other deeper reasons.

Comment author: PedroCarvalho 06 December 2012 11:26:23PM 3 points [-]

I have had this question in my mind for ages. You say that these counterfactual universes don't actually exist. But, according to Many-Worlds, don't all lawful Universes actually really really exist? I mean, isn't there some amplitude for Mr. Oswald to not have shot Kennedy, and then you get a blob where Kennedy didn't get murdered?

I've been banging my head against a wall on this and still can't come to a conclusion. Are the decoherent blobs actually capable of creating multiple histories on the observable level, up here? It looks, to me, that they should be. I mean, if these particles have each an amplitude to "be" here and there, then there is some amplitude for the combination of all particles in the Universe to correspond to a completely different macro Universe.

On the other hand, that seems to also imply that there's some amplitude for things like President Kennedy being shot, and then suddenly his wounds closed and he was up and running again. And that doesn't sound okay at all.

So... I am very, very confused.

Comment author: ArisKatsaris 12 December 2012 11:00:25PM *  2 points [-]

But, according to Many-Worlds, don't all lawful Universes actually really really exist?

Nitpick: All the Many-Worlds of QM still follow our particular set of physics. For "all lawful universes" to really really exist, you probably have to go to Tegmark IV or something like that....

Comment author: PedroCarvalho 13 December 2012 09:35:53PM 0 points [-]

Yes, I'm sorry, by "lawful" I'd meant exactly that, universes that obey our particular set of physics.

Comment author: RobbBB 07 December 2012 12:19:02AM 2 points [-]

Abstractions like probability and number are constructed by us; they don't strictly exist, but it's useful to act as though they do, since they help organize our reasoning. It could be that by coincidence that some part of the Real World corresponds precisely to the structure of our modal or mathematical reasoning; for instance, the many-worlds interpretation of QM could be true, or we could live in a Tegmark ensemble. But this would still just be an interesting coincidence. It wouldn't change the fact that our abstractions are our own; and if we discovered tomorrow that a Bohmian interpretation of QM is correct, rather than an Everettian one, it would have no foundational implications for such a high-level, anthropocentric phenomena as probability theory.

Thinking in this way is useful for two reasons. First, it insulates our logical fictions from metaphysical skepticism; our uncertainty as to the existence of a Platonic realm of Number need not undermine our confidence that 2 and 2 make 4. Second, it keeps us from being tempted to slide down the slippery slope to treating all our fictions (like currency, and intentionality, and qualia, and Sherlock Holmes) as equally metaphysically committing.

Comment author: PedroCarvalho 07 December 2012 08:13:13PM 0 points [-]

Well, whether probability and number exist or not is moot. The point of fact is that when you look at any quantum system there is a probability of finding it in any given (continuous set of) state(s) equals the squared modulus of the amplitude for it to be in such state. As mr. Yudkowsky once put, and I paraphrase, "I still want to know the nonexistent laws that coordinate my meaningless Universe".

And my point is: assuming Quantum Physics is completely correct, without us adding the additional postulates, do all combinations of universes exist, superposed to each other? That is to say: is the quantum suicide limited to 50/50 strictly quantised experiments, or does our consciousness live on in a forever branching multiverse? Sort of.

Comment author: drnickbone 10 December 2012 08:24:31PM *  1 point [-]

I have had this question in my mind for ages. You say that these counterfactual universes don't actually exist. But, according to Many-Worlds, don't all lawful Universes actually really really exist? I mean, isn't there some amplitude for Mr. Oswald to not have shot Kennedy, and then you get a blob where Kennedy didn't get murdered?

I had the same reaction... Can this be the same Eliezer who authored the sequences, and gave such strong support for the reality of Many Worlds?

I was half-expecting the other shoe to drop somewhere in the article... namely that if you are prepared to accept that the Many Worlds really exist, it makes the Great Reductionist Project a whole lot easier. Statements about causality reduce to statements about causal graphs, which in turn reduce to statements about counterfactuals, which in turn reduce to statements of actual fact about different blobs of the (real) quantum state vector. Similarly, statements about physical "possibility" and "probability" reduce to complicated statements about other blobs and their sizes as measured by the inner product on the state space.

Maybe Eliezer will be leading that way later... If he isn't I share your confusion.

Comment author: NancyLebovitz 07 December 2012 07:11:58AM 1 point [-]

Maybe the way out is that counterfactuals don't exist in their home universes.

Comment author: PedroCarvalho 07 December 2012 12:05:13AM 0 points [-]

I just read Mr. Yudkowsky's articles on Boltzmann Brains and the Anthropic trilemma... and I had thought of those questions a while ago. While they're not directly related to this comment, I guess I should comment about them here, too.

I have no problem thinking of myself as a Boltzmann Brain. Since most (if not all) such Brains will die an instant after existing, I guess my existence could be accurately described as a string of Boltzmann Brains in different regions of spacetime, each containing a small (not sure how small) slice of my existence. Perhaps they all exist at the same time. And the Anthropic Principle would explain the illusion of continuity, somewhat. My main thoughts on the Boltzmann Brain idea is that any hypothesis that has no way to be tested even in principle is equivalent to the Null hypothesis. I guess what I mean is, if I found out right now, with P ~ 1, that my existence is a string of Boltzmann Brains, that would not affect my predictions. I'm not sure I should be thinking this... because this whole matter confuses the hell out of me, but that's my current mental state.

As for the Anthropic Trilemma... well, I guess it pretty much means mr. Yudkowsky has the same doubts as I do. Very, very confusing business indeed. Sometimes I think I should just quit thinking and become a stripper. That was a joke, by the way.

Comment author: Tyrrell_McAllister 06 December 2012 12:14:48AM *  6 points [-]

(I have not yet encountered a claim to have finished Reducing anthropics which (a) ends up with only two kinds of stuff and (b) does not seem to imply that I should expect my experiences to dissolve into Boltzmann-brain chaos in the next instant, given that if all this talk of 'degree of realness' is nonsense, there is no way to say that physically-lawful copies of me are more common than Boltzmann brain copies of me.)

I think it was Vladimir Nesov who said something like the following: Anticipation is just what it feels like when your brain has decided that it makes sense to pre-compute now what it will do if it has some particular possible future experience. You should expect experiences only if expecting (i.e., thinking about in advance) those experiences has greater expected value than thinking about other things.

On this view, which seems right to me, you shouldn't expect to dissolve into Boltzmann-brain chaos. This is because you know that any labor that you expend on that expectation will be totally wasted. If you find yourself starting to dissolve, you won't look back on your present self and think, "If only I'd thought in advance about what to do in this situation. I could have been prepared. I could be doing something right now to improve my lot."

Consider an analogous situation. You're strapped to a bed in a metal box, utterly immobilized and living a miserable life. Intravenous tubes are keeping you alive. You know that you are powerless to escape. In fact, you know that you are absolutely powerless to make your life in here any better or worse. You know that, tomorrow, your captors will roll a million-sided die, with sides numbered one to a million. If the die comes up "1", you will be released, free to make the best of your life in the wide-open world. Otherwise, if any other side of the die come up, you will remain confined as you are now until you die. There will be no other chances for any change in your circumstances.

Clearly you are more likely to spend the rest of your life in the box. But should you spend any time anticipating that? Of course not. What would be the point? You should spend all of you mental effort on figuring out the best thing to do if you are released. Your expected utility is maximized by thinking only about this scenario, even though it is very improbable. Even a single thought given to the alternative possibility is a wasted thought. You should not anticipate confinement after tomorrow. You should not expect to be confined after tomorrow. These mental activities are maximally bad options for what you could be doing with your time right now.

Comment author: bryjnar 11 December 2012 01:20:11AM 0 points [-]

You've just redefined "expect" so that the problem goes away. For sure, there's no point practically worrying about outcomes that you can't do anything about, but that doesn't mean you shouldn't expect them. If you want to argue that we should use different notion than "expect", or that the practical considerations show that the Boltzmann-brain argument isn't a problem, that's fine, but this has all the benefits of theft over honest toil.

Comment author: Tyrrell_McAllister 11 December 2012 04:57:33AM *  1 point [-]

You've just redefined "expect" so that the problem goes away.

I don't believe that there is any redefinition going on here. I intend to use "expect" in exactly the usual sense, which I take also to be the sense that Eliezer was using when he wrote "I have not yet encountered a claim to have finished Reducing anthropics which ... does not seem to imply that I should expect my experiences to dissolve into Boltzmann-brain chaos in the next instant".

Both he and I are referring to a particular mental activity, namely the activity that is normally called "expecting". With regard to this very same activity, I am addressing the question of whether one "should expect [one's] experiences to dissolve into Boltzmann-brain chaos in the next instant". (Emphasis added.)

The potentially controversial claim in my argument is not the definition of "expect". That definition is supposed to be utterly standard. The controversial claim is about when one ought to expect. The "standard view" is that one ought to expect an event just when that event has a probability of happening that is greater than some threshold. To argue against this view, I am pointing to the fact that expecting an event is a certain mental act. Since it is an act, a proper justification for doing it should take into account utilities as well as probabilities. My claim is that, once one takes the relevant utilities into account, one easily sees that one shouldn't expect oneself to dissolve into Boltzmann-brain chaos, even if that dissolution is overwhelmingly likely to happen.

Comment author: bryjnar 11 December 2012 05:21:03AM *  0 points [-]

Ah, okay. You're quite right then, I misdiagnosed what you were trying to do. I still think it's wrong, though.

In particular, I don't think the "should" in that sentence works the way you're claiming that it does. In context, "Should I expect X?" seems equivalent to "Would I be correct in expecting X?" or somesuch, rather than "Ought I (practically/morally) to expect X?". English is not so well-behaved as that. I guess it kind of looks like perhaps it's an epistemic-rationality "should", but I'm not sure it's even that.

Comment author: Tyrrell_McAllister 11 December 2012 05:42:35AM *  0 points [-]

"Should I expect X?" seems equivalent to "Would I be correct in expecting X?" or somesuch...

Then my answer would be, Maybe you would be correct. But why would this imply that anthropics needs any additional "reducing", or that something more than logic + physics is needed? It all still adds up to normality. You still make all the same decisions about what you should work to protect or prevent, what you should think about and try to bring about, etc. All the same things need to be done with exactly the same urgency. Your allegedly impending dissolution doesn't change any of this.

Comment author: bryjnar 11 December 2012 06:05:26AM 0 points [-]

Right. So, as I said, you are counselling that "anthropics" is practically not a problem, as even if there is a sense of "expect" in which it would be correct to expect the Boltzmann-brain scenario, this is not worth worrying about because it will not affect our decisions.

That's a perfectly reasonable thing to say, but it's not actually addressing the question of getting anthropics right, and it's misleading to present it as such. You're just saying that we shouldn't care about this particular bit of anthropics. Doesn't mean that I wouldn't be correct (or not) to expect my impending dissolution.

Comment author: Tyrrell_McAllister 11 December 2012 07:01:58AM *  1 point [-]

it's not actually addressing the question of getting anthropics right, and it's misleading to present it as such.

I would have been "addressing the question of getting anthropics right" if I had talked about what the "I" in "I will dissolve" means, or about how I should go about assigning a probability to that indexical-laden proposition. I don't think that I presented myself as doing that.

I'm also not saying that I've solved these problems, or that we shouldn't work towards a general theory of anthropics that answers them.

The uselessness of anticipating that you will be a Boltzmann brain is particular to Boltzmann-brain scenarios. It is not a feature of anthropic problems in general. The Boltzmann brain is, by hypothesis, powerless to do anything to change its circumstances. That is what makes anticipating the scenario pointless. Most anthropic scenarios aren't like this, and so it is much more reasonable to wonder how you should allocate "anticipation" to them.

The question of whether indexicals like "I" should play a role in how we allocate our anticipation — that question is open as far as I know.

My point was this. Eliezer seemed to be saying something like, "If a theory of anthropics reduces anthropics to physics+logic, then great. But if the theory does that at the cost of saying that I am probably a Boltzmann brain, then I consider that to be too high a price to pay. You're going to have to work harder than that to convince me that I'm really and truly probably a Boltzmann brain." I am saying that, even if a theory of anthropics says that "I am probably a Boltzmann brain" (where the theory explains what that "I" means), that is not a problem for the theory. If the theory is otherwise unproblematic, then I see no problem at all.

Comment author: RobbBB 05 December 2012 01:41:47AM *  4 points [-]

I like how you frame this discussion. At this stage, I'd like to see more LessWrongers spending sleepless nights pondering how we want to renegotiate our correspondence theory to keep our theory and jargon as clean and useful as possible. Calling ordinary assertions 'true/false' and logical ones 'valid/invalid' isn't satisfactory. Not only does it problematize mixed-reference cases, but it also confusingly conflates a property of structured groups of assertions (arguments, proofs, etc.) with a property of individual assertions.

Our prototype for 'truth' is that an assertive representation co-occur in reality with the represented circumstance. Problem discourses like ethics, alethic modality, and pure mathematics seem to deviate from the correspondence-theory prototype because our confidence in their truth or falsehood outstrips our confidence in anything corresponding to their semantic content. (For those with very sparse metaphysical views, sometimes called 'noneists,' the outstripping is especially severe.) Put simply, although our 'logical' statements seem to depend on the world — at a minimum, on our linguistic choices, our derivation rules, etc. — they don't seem to depend on there being a literal worldly correlate for what they assert. Truth-conditions and representational content come apart radically.

Perhaps we should rescue the correspondence theory by denying that the correspondence is simply a matter of the asserted circumstance obtaining? "The average Australian male is 5'9"." is not true because there exists some object, the average Australian male, falling under the extension of the predicate (or bearing the property) "being 5'9"". It must be analyzed into a more complex physical statement, or a case of mixed reference. If distinguishing purely physical from mixed statements is difficult in many cases, as singling out the purely logical statements seems to be, then this gives us more pragmatic reason to relax our constraints on truth-aptness and either abandon or broaden our correspondence theory as a general theory of truth.

It should go without saying that if we adopt this approach, we need not compromise our realism; how we use the word 'truth' is a linguistic matter, not a deep metaphysical one.

Comment author: khafra 05 December 2012 03:05:14PM 1 point [-]

...although our 'logical' statements seem to depend on the world — at a minimum, on our linguistic choices, our derivation rules, etc. — they don't seem to depend on there being a literal worldly correlate for what they assert. Truth-conditions and representational content come apart radically.

What's missing from this part, to keep it from adequately addressing the question (combined with the earlier post on the nature of logic)?

To compare a mental image of high-level apple-objects to physical reality, for it to be true under a correspondence theory of truth, doesn't require that apples be fundamental in physical law. A single discrete element of fundamental physics is not the only thing that a statement can ever be compared-to. We just need truth conditions that categorize the low-level states of the universe, so that different low-level physical states are inside or outside the mental image of "some apples on the table" or alternatively "a kitten on the table".

...And thus "The product of the apple numbers is six" is meaningful, constraining the possible worlds. It has a truth-condition, fulfilled by a mixture of physical reality and logical validity; and the correspondence is nailed down by a mixture of causal reference and axiomatic pinpointing.

Comment author: RobbBB 05 December 2012 04:16:23PM *  4 points [-]
  • I. 'Valid' is a bad word for what Eliezer's talking about, because validity is a property of arguments, proofs, inferences, not of individual assertions. For now, I'll call Eliezer's validity 'derivability' or 'provability.'

  • II. Strictly speaking, is logical derivability a kind of truth, or is it an alternative to truth that sometimes gets confused with it? Eliezer seems to alternate between these two views.

  • III. Are some statements simply 'valid' / 'derivable'? Or is validity/derivability always relative to a set of inference rules (and, in some cases, axioms or assumptions)?

  • IIII. If derivability is always relativized in this way, then what does it mean to say that "The product of the apple numbers is six" is true in virtue of a mixture of physical reality and logical derivability? A different set of logical or mathematical rules would have yielded a different result. 'Logical pinpointing' is meant to solve this — there is a unique imaginary, fictional, mathematical, etc. image that we're reasoning with in every case, and 'intuitionistic real numbers' simply aren't the same objects as 'conventional real numbers,' and there simply is no such thing as 'the real numbers' absent the aforementioned specifications. Should we say, then, that truth is bivalent, whereas derivability/validity is trivalent?

Here's an example of where this sort of reasoning will lead us: First, there simply isn't any such thing as a 'continuum hypothesis;' we must exhaustively specify a set of inference rules and axioms/assumptions before we can even entertain a discrete logical claim, much less evaluate that claim's derivability. Once we have fully pinpointed the expression, say as the 'conventional continuum hypothesis' or the 'consistent Zermelo-Frankel continuum hypothesis,' we then arrive at the conclusion that the hypothesis is not true (since it is logical and not empirical); nor is it false; nor is it valid/derivable; nor is its negation valid/derivable. It is thus 'invalid' in the weak sense that it can't be derived, but is not 'invalid' in the strong sense of being disprovable. So, again, we have reason to speak of three properties (perhaps: provable, unprovable, disprovable), rather than of a bivalent 'validity.'

  • V. Supposing correspondence-conditions fix truth-conditions, what fixes the correspondence-conditions? Relatedly, what makes assertions have the particular contents and referents they do, what supplies the 'semantic glue'? And do logical or mixed-reference truths refer to anything in the world? If so, to what?
Comment author: torekp 08 December 2012 03:37:01PM 0 points [-]

I especially like point/questions V. If we abandon the correspondence theory of truth, can we duck the questions? Because answering them seems like a lot of work, and like Dilbert and his office mates, I love the sweet smell of unnecessary work.

Comment author: Ritalin 06 December 2012 10:13:14PM *  2 points [-]

Take the apples and grind them down to the finest powder and sieve them through the finest sieve and then show me one atom of sixness, one molecule of multiplication.

Discworld reference FTW. I would suspect that Pratchett's Death, being the secular humanist and life enthusiast that he is, would strongly approve of our efforts here to eventually render him irrelevant.

Comment author: shminux 05 December 2012 07:31:00PM 2 points [-]

Could the Born probabilities be basic - could there just be a basic law of physics which just says directly that to find out how likely you are to be in any quantum world, the integral over squared modulus gives you the answer? And the same law could've just as easily have said that you're likely to find yourself in a world that goes over the integral of modulus to the power 1.99999?

But then we would have 'mixed references' that mixed together three kinds of stuff - the Schrodinger Equation, a deterministic causal equation relating complex amplitudes inside a configuration space; logical validities and models; and a law which assigned fundamental-degree-of-realness a.k.a. magical-reality-fluid. Meaningful statements would talk about some mixture of physical laws over particle fields in our own universe, logical validities, and degree-of-realness.

I guess I understand better now where your dislike of the "shut up and calculate" non-interpretation of QM is coming from. You refuse to acknowledge that the Born probabilities could be a manifestation of some deeper physical law we do not yet know, and that the Schrodinger equation could be another manifestation of the same law, thus removing the need for the "third thing". The standard reaction to what I just said is "but we don't need anything else, just the Schrodinger equation", and then proceed to make extra assumptions equivalent to the Born rule, only more complicated.

Comment author: torekp 08 December 2012 04:03:34PM *  1 point [-]

In the first paragraph you quoted, EY arbitrarily and pointlessly juxtaposes two different questions. I say "pointlessly" charitably, because if there is a point, it's a bad one, to (guilt-by-)associate an affirmative answer to the first, with an affirmative answer to the second.

Could the Born probabilities be basic? "Could" would seem best interpreted here as "formulable consistently with the two-factor Great Reductionist approach." "Basic" I'll take as relative to a model: if a law is derived in the model, it's not basic. Now that we know what the question is, the answer is: sure, why not? Physical laws mention "electric charge", "time", "distance"; adding "probability" doesn't seem to break anything, as long as the resulting theory is testable. That basically probabilistic theory might not be the most elegant, but that's a different argument. And there's no need to top probabilities with fundamental-degree-of-realness sauce.

Comment author: shminux 08 December 2012 07:19:09PM 0 points [-]

Physical laws mention "electric charge", "time", "distance"; adding "probability" doesn't seem to break anything, as long as the resulting theory is testable.

He is not an instrumentalist, so he finds this approach (anything that helps one make good predictions goes) aesthetically unsatisfying.

Comment author: torekp 12 December 2012 02:38:16AM *  0 points [-]

I'm not saying or implying that "anything that helps one make good predictions, goes". I really don't think instrumentalism is relevant here; if we take it off the table as an option, there still doesn't seem to be any reason to disprefer a theory that posits "objective probability" to one that posits "electric charge", aside from the overall elegance and explanatory power of the two theories. Which are reasons to incline to believe that a theory is true, I take it, not just to see it as useful.

Comment author: BobTheBob 08 December 2012 09:20:50PM 3 points [-]

This is an interesting post but, I have to say, kind of frustrating. I have tried to follow the discussions between Esar and RobbBB and your substantial elucidation as well as many other great comments, but I remain kind-of in the dark. Below are some questions which I had, as I read.

This question doesn't feel like it should be very hard.

What question? What exactly is the problem you are purporting to solve, here? If it is, "What is the truth condition of 'If we took the number of apples in each pile, and multiplied those numbers together, we'd get six.'", then doesn't Tarski's disquotation schema give us the answer?

Navigating to the six requires a mixture of physical and logical reference

Not sure why you obscure matters with idiosyncratic metaphors like 'navigating to the six', but never mind. Can we infer from the distinction between logical and physical reference that there is a distinction between logical and physical truth? It appears you countenance the Analytic/Synthetic distinction - precisely the distinction which is usually considered to have undone logical positivism. Do you have a preferred response to Quine's famous argument in 'Two Dogmas of Empiricism', or do you have a reason for thinking you are immune to it? I think you think you aren't doing philosophy, so it doesn't apply, but then I really don't know how to understand what you're saying. If your problems are just computational, then surely you're making matters much harder for yourself than they should be (not that computational problems aren't sometimes very hard).

Next we have to call the stuff on the table 'apples'. But how, oh how can we do this...?

How about by saying "Those are apples"? What exactly is the problem, here?

...when grinding the universe and running it through a sieve will reveal not a single particle of appleness?

Here's my best guess at what is exercising you. You reason that only those properties needed to account for the constitution and behaviour of the smallest parts of matter are real, that being an apple is not among them, and hence that being an apple is not a real property. Assuming this guess is right, what exactly is your reason for accepting the first premise? It is not immediately obvious, though I know there are traditionally different reasons. The reason will inform the adequacy of your answer.

Standard physics uses the same fundamental theory to describe the flight of a Boeing 747 airplane, and collisions in the Relativistic Heavy Ion Collider. Nuclei and airplanes alike, according to our understanding, are obeying special relativity, quantum mechanics, and chromodynamics.

So far so good...

We also use entirely different models to understand the aerodynamics of a 747 and a collision between gold nuclei in the RHIC. A computer modeling the aerodynamics of a 747 may not contain a single token, a single bit of RAM, that represents a quark. (Or a quantum field, really; but you get the idea.)

Nothing controversial here, but it of course has nothing to do with our understanding of the problem. If the understanding is correct, the problem exists regardless of whether anyone or thing ever imagines or represents or refers to apples or anything else. To introduce representations and models into the discussion is only to confuse matters, no?

So is the 747 made of something other than quarks?

Where does this question come from? If my guess about the problem is correct, it is irrelevant. It may be that the property of being a 747 (apple) is not identical to the property of being in any very complicated way composed of quarks, bosons and leptons, even though a given 747 (apple) is made only of these particles. The (philosophical) thesis about properties is different than the scientific thesis about the constitution of physical objects.

No, we're just modeling the 747 with representational elements that do not have a one-to-one correspondence with individual quarks. Similarly with apples.

Please clarify - what precisely does the relation between a computer model of a 747 and a 747 have to do with the metaphysics of properties?

To compare a mental image of high-level apple-objects to physical reality,

Can you say what you mean by this? For myself, this is someting I only ever do very rarely -conjure a mental image, then see how it agrees or differs from what I'm looking at. To be sure, sets of neurons in my brain are being activated all the time by patterns of light hitting my retinas, but there's a lot of explanatory distance to cover to show these (the story about neural events and the story about images) are the same thing. In any case, this seems entirely irrelevant to the present concerns.

for it to be true

Are mental images the sorts of thngs which can be true (in the sense in which a sentence or proposition can be, as opp. merely accurate)? Suppose I have a mental picture of a certain cat on a certain table and that the cat is indeed on the table. Is my mental image true? Even if the cat in my image is the wrong colour? Or is sitting when the cat is standing? As far as I can see this isn't just nit-picking. You have some kind of AI model which involves mental images and which you seem to think needs a semantic theory, and it's just not clear how it all fits together.

...doesn't require that apples be fundamental in physical law.

If my guess is correct, your answer to the problem as far as I can see is something like "The problem is not a problem".

A single discrete element of fundamental physics is not the only thing that a statement can ever be compared-to. We just need truth conditions that categorize the low-level states of the universe, so that different low-level physical states are inside or outside the mental image of "some apples on the table" or alternatively "a kitten on the table".

Can you give an example of a low-level state being 'inside a mental image' of "some apples on the table"? I really don't know what this means.

Having gone through this once, here's a second pass at a gloss. You accept, reasonably, that "That is an apple" is true in English iff that (pointing to a certain apple) is an apple. The referent of the "that" we can take to be a certain object. The question arises, however, as to what the referent or other semantic value is of "is an apple". Plausibly, it is the property of being an apple. But, we may reasonably ask, what sort of thing is being and apple? I understand your answer is as follows:

Just as an individual apple is nothing more than a quite large number of quarks and leptons and bosons interestingly assembled, being an apple is nothing more than being a quite large number of quarks and leptons and bosons assembled in a certain interesting way.

Is this roughly a fair understanding? If so, please consider:

1) You will need to augment your story to include so-called etiology. The property of being a 10-dollar bill is not equivalent to the property of being in a certain way composed of matter - causal origin/history matters, too (perfect counterfeits).

2) The problem of vagueness often seems like a paradigm of philosophical futility but it is a real problem. Suppose you could cross-breed apples and pears, and have a spectrum of individuals ranging from unproblematic apple to unproblematic pear (= non-apple). What will the truth-condition be of the statement 'That is an apple', pointing to the piece of fruit in the middle? Do you give up on bivalence, or do you say that the statement is determinately true or false, but there are deep epistemological problems? Neither answer seems satisfactory, and where you come down may affect your theory.

3) If this story is correct, it will presumably apply to the whole very large hierarchy of properties, ranging from being a quark through being a proton and being a carbon atom up to being an apple and beyond. And the high-level properties will have at a minimum to be disjunctions of lower properties, even to accomodate such mundane facts as the existence of both green and red apples. And you may find ultimately that what is in question is more like a family-resemblance relation among the cases which constitute being an apple (if not apples, then tables and 747s, very likely). And then aren't you in danger simply of laboriously re-capitulating the history of 20th c. philosophical thought on the subject?

This is all philosophy, which you've repeatedly said you aren't interested in doing. But that's what you're doing! If you're just doing AI, you really shouldn't be wasting your time on these questions, surely. Research into neural nets is already making great progress on the question of how we make the discriminations we do. Why isn't that enough for your purposes?

A last thought: there's something of a debate on this site about the value of traditional philosophy. I think it has value, a big part of which is that it encourages people to think carefully and to express themselves precisely. I don't claim always to be as careful or precise as I should be, but these are values. Doing analytic philosophy is some of the best rationality training you can get.

Comment author: DSimon 08 December 2012 09:54:32PM *  2 points [-]

Regarding your point numbered 1 specifically: the causal history of matter is considered here as part of its physical properties in a block universe, so this objection doesn't apply. See the older sequence article Timeless Physics for more on this.

Regarding points 2 and 3: The OP is saying that for something to be an apple means that its low-level physical state matches some pattern, but not necessarily that the pattern matching function must return a strict True or False; there are fuzzy pattern matching functions as well. The older sequence article Similarity Clusters goes into this in more detail.

On the other hand, your objections are totally legit within the context of this article and its examples alone, and as an introductory article that's a fine and appropriate context to be working from. Maybe the article would be improved by some footnotes and/or appropriate links? Then again, it's already pretty long.

On the other other hand, as an introductory article its purpose is only to introduce what reductionism is and get people to grips with the notion of different levels of abstraction. If philosophical arguments are being made about e.g. what is "real", or more subtly about what makes for an appropriate definition of a word like "apple", then they aren't being made here, but in the articles that depend on this one. "Lying to children" and all that.

Comment author: shminux 05 December 2012 01:22:20AM 3 points [-]

Not sure how good an example apple multiplication is, given that if you multiply 2 apples by 3 apples, you are supposed to get 6 square apples.

Comment author: Eliezer_Yudkowsky 05 December 2012 01:50:18AM 15 points [-]

Hence my careful specification that you're multiplying the numbers, not the piles.

Comment author: johnswentworth 05 December 2012 02:25:56AM 10 points [-]

I found the use of multiplication particularly useful, since it forced the reader to pay attention to the physical/logical distinction. If, say, addition had been used, then a determined reader could try to use physical constraints alone (though they would be cheating).

Comment author: faul_sname 05 December 2012 03:51:28AM *  1 point [-]

If we assume that the 5 apples are spherical, and we cut the largest square sections possible out of each of them (leaving the top and bottom alone, as that doesn't affect whether the shape is a square when viewed from the top down), it turns out that these new squared apples have a volume of about 0.77 times that of a spherical apple. That means that your 2 round apples and your 3 round apples become about 6.49 squared apples. Rounding down, that is, in fact, 6 square apples.

But I do think the illegal operation was kind of the point. It shows that not all mathematical operations can be strictly reduced to physical objects (well, outside of the substrate that's doing the computing, obviously).

Edit: it was

Comment author: army1987 05 December 2012 04:22:24PM 0 points [-]

You might want to add some kind of smiley at the end of the first paragraph. (I didn't downvote, but I suspect that's the reason why someone did.)

Comment author: Bugmaster 05 December 2012 09:50:54PM 2 points [-]

I am not convinced -- by this article, at least -- that there could only be two kinds of stuff. It sounds like the answer to the question, "why two and not one or possibly three ?" is, "because I said so", and that's not very convincing.

I am also not entirely sure what the Great Reductionist Project is, or why it's important.

Note that I'm not arguing against reductionism, but solely against this post.

Comment author: Johnicholas 05 December 2012 01:04:23PM 1 point [-]

It may not be possible to draw a sharp line between things that exist from the things that do not exist. Surely there are problematic referents ("the smallest triple of numbers in lexicographic order such that a^3+b^3=c^3", "the historical jesus", "the smallest pair of numbers in lexicographic order such that a^3+24=c^2", "shakespeare's firstborn child") that need considerable working with before ascertaining that they exist or do not exist. Given that difficulty, it seems like we work with existence explicitly, as a theory; it's not "baked in" to human reasoning.

Guy Steele wrote a talk called "Growing a Language", where one of his points is that building hooks (such as functions) into the language definition to allow the programmer to grow the language is more important than building something that is often useful, say, complex numbers or a rich collection of string manipulation primitives. Maybe talking about the structure of "theories of X" would be valuable. Perhaps all theories have examples (including counterexamples as a specific kind of example) and rules (including definitions as a specific kind of example) - thats the kind of thing that I'm suggesting might be more like a hook.

Comment author: Armok_GoB 05 December 2012 03:57:42PM 1 point [-]

Tangential: I keep not understanding counterfactuals intuitively, not because of the usual reason, but simply because if I take my best model of the past and rerun it towardsthe future I do not arive at the present due to stochastic and chaos elevents.

Aka, trying to do the standard math: I throw a 100 sided dice, it comes out 73, "If 2+2 were equal to 4, the dice would with 99% certainty have come out 73".

Comment author: torekp 08 December 2012 04:14:05PM 3 points [-]

If 2+2 were equal to 4, the dice would with 99% certainty [not] have come out 73

The statement is true, but because making a statement in a conversation is normally taken to have a point, nobody would ever say such a thing. If it rings false to your ears, that's your social instincts rightly warning you that making such a statement would be likely to deceive someone.

Compare: my super-smart friend is studying for a test. I know he'll ace it no matter what. I wouldn't tell him "if you go to bed now and get some sleep you'll ace it tomorrow", and I wouldn't tell him "if you study all night you'll ace it", despite both of those being true. In either case he would think the first part of my statement was relevant.

Comment author: Armok_GoB 08 December 2012 08:50:56PM 1 point [-]

Then how can anyone meaningfully talk about "what would have happened if X had happened instead of Y, Z years ago", when there'd be billions of changes due to randomness vastly larger than the kind of things humans tend to respond to that type of question with, completely drowning them out?

Comment author: [deleted] 05 December 2012 12:57:50PM 1 point [-]

Great post as usual, Eliezer! I have to admit that I never thought of logical and causal references being mixed before, but truly that is often exactly how we use them.

I have one question, though: I read through the quantum physics sequence, and I just don't understand - why are the Born probabilities such a problem? Aren't there just blobs of amplitude decohering? Is the problem that all the decoherence is already predicted to happen, without implying the Born rule? If someone could clarify this for me, I'd greatly appreciate it.

Comment author: PedroCarvalho 06 December 2012 11:21:02PM 2 points [-]

I am not sure I am correct, but if I'm not mistaken, the problem with the Born rule is that no one so far has successfully (in the eyes of their peer physicists) proven they must be true. As in, they're additional. If you go by the standard Copenhagen interpretation, since Collapse is already an arbitrary additional rule, it already sort of contains the Born probabilities: they're just the additional rules that additionally condition how Collapse happens. But any other theories that remove objective, additional Collapse from the picture have this big problem: why, oh, WHY do we get the Born probabilities?

Furthermore, we have an even more interesting question: what do they even mean?! Suppose you (temporarily) accept the Born probabilities. What are they probabilities of? Meaning: if there is a 75% chance that you will observe a photon polarised in a given direction, what does that mean, in the grand scheme? Are you divided into 100 copies of you, and 75 of them observe such polarisation, while 25 of them don't?

That's... pretty much it. I hope I could help.

Comment author: JMiller 05 December 2012 02:07:24AM *  1 point [-]

I am somewhat confused about the nature of logical axioms. They are not reducible to physical laws, and physical laws are not reducible to logic. So then, it what sense are they (axioms) real? I don't think you are saying that they are "out there" in some Platonic sense, but it also seems like you are taking a realist or quasi-empirical approach to math/logic.

Comment author: Armok_GoB 05 December 2012 04:35:13PM -3 points [-]

Your assumption that physical laws are not reducible to logic is false. http://arxiv.org/abs/0704.0646

Comment author: shminux 05 December 2012 10:44:38PM 3 points [-]

This is extremely controversial, so I'd not use the word "false" here.

Comment author: JMiller 05 December 2012 04:44:32PM 2 points [-]

I don't have time to read this this week, but when I do I will get back to you. Thanks for the article.

Comment author: JMiller 15 December 2012 05:35:24AM 0 points [-]

Thanks for the paper! I have started to read this and am admittedly overwhelmed. I think I understand the concept, but without the ability to understand the math, I feel limited in my scope to comprehend this. Would you be able to give my a brief summary of why we should accept MUH and why it is controversial?

Comment author: Armok_GoB 15 December 2012 05:40:43PM *  1 point [-]

We should believe MUH because it's mathematically impossible to consistently believe in anything that's not maths, because beliefs are made of maths and can't refer to things that are not maths.

It' controversial because humans are crazy, and can't ignore things genetically hard coded into their subconscious no matter how little sense it makes.

RDIT: Appears I were stupid an interpreted your question literally instead of trying to make an actual persuasive explanation.

Can't really help you with that, I absolutely suck at explain things, especially things I see as self evident. I literally can not imagine what it being any other way would even mean, so I can't explain how to get from there to here.

Comment author: JMiller 16 December 2012 06:27:35PM 1 point [-]

I appreciate your attempt to try though. Thanks.

Comment author: ArisKatsaris 15 December 2012 05:56:50PM *  1 point [-]

beliefs are made of maths and can't refer to things that are not maths.

...to me that sounds like saying "words are made of letters and can't refer to things that are not letters, therefore e.g. trees and clouds must be made of letters." It sounds like a map-territory confusion of insane degree.

The Mathematical Universe Hypothesis may be true, but this argument doesn't really work for me.

Comment author: Armok_GoB 16 December 2012 01:42:17AM 0 points [-]

Correct, I've edited my post to clarify.

Comment author: Peterdjones 15 December 2012 05:44:03PM 1 point [-]

We should believe MUH because

it's mathematically impossible to consistently believe in anything that's not maths,

I can see no evidence fror that.

because beliefs are made of maths

or that.

and can't refer to things that are not maths.

or that. I also don't see how the conclusion follows even if they are all true.

Comment author: Armok_GoB 16 December 2012 01:41:47AM 0 points [-]

Yea I were stupid, edited my post.

Comment author: Benito 08 December 2012 02:04:59PM *  0 points [-]

Logical Axioms are the rules that decide what can and can't happen. Then, our physical world is one application of these to some starting physical position (and that may be logical defined too, read this post, or Good and Real).

Logic is useful when we have uncertainty. If we are unsure about a certain variable, we can extrapolate to how the future will be given the different possibilites - the different variables that are logically consistent within a causal universe that fits with everything else we know. Of course, if we had no causal knowledge whatsoever, then we'd not have anything with which to apply logic (kinda like this post, with causal reference being emotions, and logic being logic).

So, I'm saying that logic can define how everything that could be would work, which we deduce from our universe's laws. If we have uncertainty, then logic defines the possibilites. If we pretend to have only the knowledge of one law, like '1 + 1 = 2', then we can find out more using logic. And this is the study of mathematics.

Comment author: ThrustVectoring 05 December 2012 02:10:49AM 0 points [-]

I think this may have been answered earlier. They are a set of ways you think a certain class of problem works. They're very much an element of your mental model of reality.

In other words, math (or logical axioms) are what adding two pebbles and three pebbles has in common with adding two apples and three apples.

Comment author: JMiller 05 December 2012 02:22:35AM 0 points [-]

Thank you. In that case, does math rely on at least one particular agent or computer having some [true] model that 2+3 = 5?

Comment author: ThrustVectoring 05 December 2012 02:49:34PM 1 point [-]

Uhm, not really. I'm not entirely sure what you mean by "math relies on things doing math". Math isn't about the thinking apparatus doing math. It's a way of systematically reducing the complexity of your mental models - it replaces adding pebbles and adding apples with just adding.

If you imagine a universe with 4 particles in it, then 2+3 is still 5.

Comment author: JMiller 05 December 2012 03:37:19PM 0 points [-]

I found Eliezer's post "Math is Subjectively Objective" which explains his position very clearly. Thanks for your help.

Comment author: Peterdjones 05 December 2012 04:21:08PM 1 point [-]

I found Eliezer's post "Math is Subjectively Objective" which explains his position very clearly.

No it doesn't, since it ends "Damned if I know."

Comment author: JMiller 05 December 2012 04:26:34PM 3 points [-]

Right, which explains his position: math is real and 2+3 really is 5, but he does not know what that means, or where that is true.

You are right though, it isn't a fully fleshed out account. All I said is that it explains his position clearly, not that his position itself is perfectly clear.

Comment author: DaFranker 05 December 2012 04:56:59PM *  0 points [-]

I think this question is somewhat ambiguous; you've gotten two correct answers that say "contradicting" (different) things and apparently answer different questions.

When you say math, are you talking about the way apples and stones interact and the states of the universe afterwards when the universe performs "operations" on them? If so, then math is agent-independent, as the world-state of 2+3 apples will be five apples regardless of the existence of some agent performing "2+3=5" in that universe.

If you're talking about the existence of the "rules of mathematics", our study of things and of counting, along with the knowledge and models that said abstract study implies, then it does rely on agents having 2+3=5 models, because otherwise there's just a worldstate with two blobs of particles somewhere, three blobs of particles elsewhere, and then a worldstate that brings the blobs together and there's a final worldstate that doesn't need "2+3=5" to exist, but requires an agent looking at the apples and performing "mathematics" on their model of those blobs of particles in order to establish the model that two and three apples will be five apples.

In other words, what-we-know-as "mathematics" would not have been invented if there were no agent using a model to represent reality, as mathematics are abstract methods of description. However, the universe would continue to behave in the same manner whether we invented mathematics or not, and as such the behaviors implied by mathematics when we say "2+3 apples = 5 apples" are independent of agents.

Comment author: JMiller 05 December 2012 06:18:38PM 0 points [-]

So when an agent or computing device performs an operation on real numbers, say division of 1200 by 7, that result is real, even though the instance of this division requires the agent to do it? The answer IS the only answer, but without an agent, there would not be a question in the first place?

Comment author: DaFranker 05 December 2012 06:29:12PM 0 points [-]

that result is real

That result is logically valid and consistent, but does not have any new physical real-ness that it didn't already have - that is, its correlation and systematic consistency with the rules of how the universe works.

Otherwise, yes, exactly.

Comment author: Alexei 08 December 2012 12:19:35AM 0 points [-]

This is just the same sort of problem if you say that causal models are meaningful and true relative to a mixture of three kinds of stuff, actual worlds, logical validities, and counterfactuals, and logical validities.

You have a typo there, I think. "Logical validities" appears twice. If it's not a type, the sentence is very unclear.