p(a,b,c) = p(a)p(b)p(c) isn't a statement of uncorrelatedness but of independence. Using the term "uncorrelated" with that meaning might be defensible but probably merits mention as something not-mainstream.
It's worth noting that the Center for Applied Rationality ran the June minicamp experiment using a standard but unusual statistical method of sorting applicants into pairs that seemed of roughly matched prior ability / prior expected outcome, and then flipping a coin to pick one member of each pair to be admitted or not
As an aside, if you're interested in looking up more about this nifty experimental design trick, the magic keyword is "blocking". The idea of randomized block designs dates back to Fisher.
As previously stated, the take on causality's math is meant to be academically standard; this includes the idea of decomposing the X(i) into deterministic F(i) and uncorrelated U(i).
I haven't particularly seen anyone else observe that claiming you know about X without X affecting you, you affecting X, or X and your belief having a common cause, violates the Markov condition on causal graphs.
I haven't actually seen anyone cite the Markov condition as a reply to the old "What constitutes randomization?" debates I've glimpsed, but I would be genuinely surprised if Pearl & co. hadn't pointed it out by now - my understanding is that he's spending most of his time evangelizing causality to experimental statisticians these days. It seems pretty obvious once you have causal models as a background.
The concept of "separate magisteria" is as old as scientific critique of religion, but the actual phrase was coined by Stephen Jay Gould (speaking favorably of the separation, natch). So far as I know, the concept of anti-epistemology is an LW original; likewise the view that causality is more general than anyone trying to separate their magisterium would ...
Whatever people try to imagine that science supposedly can't analyze, it just ends up as more "stuff that makes stuff happen and happens because of other stuff".
I think said people would object to this - e.g. "God certainly isn't stuff, God is metaphysical!" This, of course, is not problem for causal diagrams. The math allows you to have arrows from metaphysical stuff to physical stuff, which allows you to see occam's razor visually. But it's interesting to think about how to best counter this argument when you're trying to convince your opponent and not just yourself.
Clearly, using 1-0-1-0-1-0 on a list of patients in alphabetical order isn't random enough... or is it?
It's not, if only because the people implementing it can guess it: a textbook I read on doing medical trials mentioned that this procedure was done in medical trials, and it led to tampering where doctors would send the patients they liked better or were sicker or whatever to the 'right' trial arm.
Something like that. There are a lot of ways to tamper with this: participation is voluntary, of course, so if a patient would 'benefit' from being randomized to the 'right' arm, you'd encourage them to do it, while if they weren't, you'd encourage them to drop out (and maybe get the tested treatment themselves!). You'd filter the list in the first place, or use alternative names (My legal first name starts with one letter but I always go by a version of my middle name which starts with a different letter: which version does the doctor write down?). And so on.
One interesting example, from a retrospective:
It took only a few months to accumulate the required experience in the two hospitals (Reese et al. 1952). Allocation to ACTH or no ACTH was decided by drawing marbles from a jar containing an equal number of white and blue marbles: one morning, when a new infant became eligible for enrollment, I noticed that our head nurse shook the jar vigorously, turned her head away, pulled a marble out (just as she had been instructed); but because she didn't like the 'assignment', she put the marble back, shook the jar again, and pulled out the color that agreed with her bias! The importance of Bradford Hill's precaution in Britain's famous streptomycin trial to conceal the order of assignment in sealed envelopes was immediately obvious!
Twice in this article, there are tables of numbers. They're clearly made-up, not measured from experiment, but I don't really understand exactly how made-up they are - are they carefully or casually cooked?
Could people instead use letters (variables), with relations like 'a > b', 'a >> b', 'a/b > c' and so on in the future? Then I could understand better what properties of the table are intentional.
Commenter HistoricalLing does have a point. Katsuki Sekida explains:
"Now, 'Mu' means 'nothing' and is the first koan in Zen. You might suppose that, as you sit saying 'Mu', you are investigating the meaning of nothingness. But that is quite incorrect. It is true that your teacher, who has instructed you to work on Mu, may repeatedly say to you, 'What is Mu?' 'Show me Mu,' and so on, but he is not asking you to indulge in conceptual speculation. He wants you to experience Mu. And in order to do this, technically speaking, you have to take Mu simply as ...
I think the division into problems and exercises usually seen in mathematics texts would be useful: A task is considered an exercise if it's routine application of previous material, it's a problem if it requires some kind of insight or originality. So far most of the Koans have seemed more like problems than like exercises, but depending on content both may be useful. I might be slightly biased towards this as I greatly enjoy mathematics texts and am used to that style.
"Problem" suggests something different in philosophy than in math. A philosophy "problem" is a seeming dilemma, e.g. Gettier, Newcomb's, or Trolley. So I'd suggest "exercise" here.
"Exercise" dominates "kōan" in that both have the sense of something to stop and think about and try to solve, but ① "exercise" avoids the misconstrual of Zen practice (the purpose of a Zen kōan is not to come up with a solution, nor to set up for an explanation), ② the Orientalism (the dubiosity of saying something in Japanese to make it sound 20% cooler), and ③ the distraction of having to explain what a kōan is to those who don't know the word.
EDIT: The claim that a purpose of a Zen kōan is not to come up with a solution appears to be a matter of disagreement, so discount ①. I think ② and ③ stand, though.
When I google "koan", the first result is Wikipedia which says a koan is "a story, dialogue, question, or statement, which is used in Zen practice to provoke the "great-doubt", and test the students progress in Zen practice". Very Zen, that supports my side. The second result is Merriam-Webster's dictionary, which says a koan is "a paradox to be meditated upon that is used to train Zen Buddhist monks to abandon ultimate dependence on reason". My side. The third result is for a page titled "101 Zen Koans", which again supports my belief.
Eliezer has a history of associating mysticism with rationality, as well.
My personal concern is that using words wrong is annoying because I don't like people mucking up my conceptual spaces. I can't disassociate koans from mysticism and riddles, which makes it awkward and aesthetically unpleasing for me to approach problems of rationality from a "koan".
That said, it's probably too late to change the format of the problems in this current sequence. But I'd like it to never happen again after this gets done.
You seem to be exaggerating the generality of the causal Markov condition (CMC) when you say it is deeper and more general than the second law of thermodynamics. In a big world, failures of the CMC abound. Let's say the correlation between the psychic cousin's predictions and the top card of the deck is explained by the person performing the test being a stooge, who is giving some non-verbal indication to the purported psychic about the top card. So here we have a causal explanation of the correlation, as the CMC would lead us to expect. But since we are i...
I am really enjoying these causality posts. Thank you for them and for the skillful writing that makes them so readable.
Um, let's see if I get this (thinking to myself but posting here if anyone happens to find this useful - or even intelligible)...
claiming you know about X without X affecting you, you affecting X, or X and your belief having a common cause, violates the Markov condition on causal graphs
The causal Markov condition is that a phenomenon is independent of its noneffects, given its direct causes. It is equivalent to the ordinary Markov condition for Bayesian nets (any node in a network is conditionally independent of its nondescendents, given its parents) w...
More generally, for me to expect your beliefs to correlate with reality, I have to either think that reality is the cause of your beliefs, expect your beliefs to alter reality, or believe that some third factor is influencing both of them.
I can construct examples where for this to be true requires us to treat mathematical truths as causes. Of course, this causes problems for the Bayesian definition of "cause".
To answer your discussion about randomizing the control groups and experimental groups- you don't use randomness or noise to divide those groups. You divide the population for study into the number of groups you need, and make that division such that those groups are as close to identical as possible, using all of the data you have on all of them.
Thermal noise and pseudo-random numbers can be used to break ties, but only because if there were any known distinction between the two outcomes, the classification would be deterministic.
"universe is a connected fabric of causes and effects."
I do not think that the universe as a whole is one fabric of causes and effects. There are isolating layers of randomness and chaos upon which there are new layers of emergence. This is why we can model at all without having one unified model.
"Every causally separated group of events would essentially be its own reality."
Places outside our solar system are their own realities in that sense. We have no effect there. Only maybe someone is there to amplify our radio signals.
Having spent a regrettably large amount of time on forums where the 'magisteria' type questions were had, I think that you're representing the 'outside of science' position slightly unfairly. Obviously, it often tries to have its cake and eat it. But you're substituting 'standard rationality', or perhaps 'questions of cause and effect' for 'science'. Some magisteria-types would say that there are direct causal effects from God or ghosts, but that these do not manifest with the regularity of things that you're likely to be able to find through scientific ex...
Any time there's a noun, a verb, and a subject[sic], there's causality.
Counterexamples "I know this." "Rational people with the same information cannot reasonably disagree about their conclusions." "General and Special Relativity both require that observers in different reference frames measure the length of an artifact differently."
I think you might have meant is "Any time that a concrete subject takes an action with a direct object", there's causality; there's probably a more general form.
I know that the top ca...
Followup to: Causality: The Fabric of Real Things
Previous meditation:
"You say that a universe is a connected fabric of causes and effects. Well, that's a very Western viewpoint - that it's all about mechanistic, deterministic stuff. I agree that anything else is outside the realm of science, but it can still be real, you know. My cousin is psychic - if you draw a card from his deck of cards, he can tell you the name of your card before he looks at it. There's no mechanism for it - it's not a causal thing that scientists could study - he just does it. Same thing when I commune on a deep level with the entire universe in order to realize that my partner truly loves me. I agree that purely spiritual phenomena are outside the realm of causal processes that can be studied by experiments, but I don't agree that they can't be real."
Reply:
Fundamentally, a causal model is a way of factorizing our uncertainty about the universe. One way of viewing a causal model is as a structure of deterministic functions plus uncorrelated sources of background uncertainty.
Let's use the Obesity-Exercise-Internet model (reminder: which is totally made up) as an example again:
We can also view this as a set of deterministic functions Fi, plus uncorrelated background sources of uncertainty Ui:
This says is that the value x3 - how much someone exercises - is a function of how obese they are (x1), how much time they spend on the Internet (x2), plus some other background factors U3 which don't correlate to anything else in the diagram, all of which collectively determine, when combined by the mechanism F3, how much time someone spends exercising.
There might be any number of different real factors involved in the possible states of U3 - like whether someone has a personal taste for jogging, whether they've ever been to a trampoline park and liked it, whether they have some gene that affects exercise endorphins. These are all different unknown background facts about a person, which might affect whether or not they exercise, above and beyond obesity and Internet use.
But from the perspective of somebody building a causal model, so long as we don't have anything else in our causal graph that correlates with these factors, we can sum them up into a single factor of subjective uncertainty, our uncertainty U3 about all the other things that might add up to a force for or against exercising. Once we know that someone isn't overweight and that they spend a lot of time on the Internet, all our uncertainty about those other background factors gets summed up with those two known factors and turned into a 38% conditional probability that the person exercises frequently.
And the key condition on a causal graph is that if you've properly described your beliefs about the connective mechanisms Fi, all your remaining uncertainty Ui should be conditionally independent:
or more generally
And then plugging those probable Ui into the strictly deterministic Fi should give us back out our whole causal model - the same joint probability table over the observable Xi.
Hence the idea that a causal model factorizes uncertainty. It factorizes out all the mechanisms that we believe connect variables, and all remaining uncertainty should be uncorrelated so far as we know.
To put it another way, if we ourselves knew about a correlation between two Ui that wasn't in the causal model, our own expectations for the joint probability table couldn't match the model's product
and all the theorems about causal inference would go out the window. Technically, the idea that the Ui are uncorrelated is known as the causal Markov condition.
What if you realize that two variables actually are correlated more than you thought? What if, to make the diagram correspond to reality, you'd have to hack it to make some Ua and Ub correlated?
Then you draw another arrow from Xa to Xb, or from Xb to Xa; or you make a new node representing the correlated part of Ua and Ub, Xc, and draw arrows from Xc to Xa and Xb.
(Or you might have to draw some extra causal arrows somewhere else; but those three changes are the ones that would solve the problem most directly.)
There was apparently at one point - I'm not sure if it's still going on or not - this big debate about the true meaning of randomization in experiments, and what counts as 'truly random'. Is your randomized experiment invalidated, if you use a merely pseudo-random algorithm instead of a thermal noise generator? Is it okay to use pseudo-random algorithms? Is it okay to use shoddy pseudo-randomness that a professional cryptographer would sneer at? Clearly, using 1-0-1-0-1-0 on a list of patients in alphabetical order isn't random enough... or is it? What if you pair off patients in alphabetical order, and flip a coin to assign one member of each pair to the experimental group and the control? How random is random?
Understanding that causal models factorize uncertainty leads to the realization that "randomizing" an experimental variable means using randomness, a Ux for the assignment, which doesn't correlate with your uncertainty about any other Ui. Our uncertainty about a thermal noise generator seems strongly guaranteed to be uncorrelated with our uncertainty about a subject's economic status, their upbringing, or anything else in the universe that might affect how they react to Drug A...
...unless somebody wrote down the output of the thermal noise generator, and then used it in another experiment on the same group of subjects to test Drug B. It doesn't matter how "intrinsically random" that output was - whether it was the XOR of a thermal noise source, a quantum noise source, a human being's so-called free will, and the world's strongest cryptographic algorithm - once it ends up correlated to any other uncertain background factor, any other Ui, you've invalidated the randomization. That's the implicit problem in the XKCD cartoon above.
But picking a strong randomness source, and using the output only once, is a pretty solid guarantee this won't happen.
Unless, ya know, you start out with a list of subjects sorted by income, and the randomness source randomly happens to put out 111111000000. Whereupon, as soon as you look at the output and are no longer uncertain about it, you might expect correlation and trouble. But that's a different and much thornier issue in Bayesianism vs. frequentism.
If we take frequentist ideas about randomization at face value, then the key requirement for theorems about experimental randomization to be applicable, is for your uncertainty about patient randomization to not correlate with any other background facts about the patients. A double-blinded study (where the doctors don't know patient status) ensures that patient status doesn't correlate with the doctor's beliefs about a patient leading them to treat patients differently. Even plugging in the fixed string "1010101010" would be sufficiently random if that pattern wasn't correlated to anything important; the trouble is that such a simple pattern could very easily correlate with some background effect, and we can believe in this possible correlation even if we're not sure what the exact correlation would be.
(It's worth noting that the Center for Applied Rationality ran the June minicamp experiment using a standard but unusual statistical method of sorting applicants into pairs that seemed of roughly matched prior ability / prior expected outcome, and then flipping a coin to pick one member of each pair to be admitted or not admitted that year. This procedure means you never randomly improbably get an experimental group that would, once you actually looked at the random numbers, seem much more promising or much worse than the control group in advance - where the frequentist guarantee that you used an experimental procedure where this usually doesn't happen 'in the long run', might be cold comfort if it obviously had happened this time once you looked at the random numbers. Roughly, this choice reflects a difference between frequentist ideas about procedures that make it hard for scientists to obtain results unless their theories are true, and then not caring about the actual random numbers so long as it's still hard to get fake results on average; versus a Bayesian goal of trying to get the maximum evidence out of the update we'll actually have to perform after looking at the results, including how the random numbers turned out on this particular occasion. Note that frequentist ethics are still being obeyed - you can't game the expected statistical significance of experimental vs. control results by picking bad pairs, so long as the coinflips themselves are fair!)
Okay, let's look at that meditation again:
"You say that a universe is a connected fabric of causes and effects. Well, that's a very Western viewpoint - that it's all about mechanistic, deterministic stuff. I agree that anything else is outside the realm of science, but it can still be real, you know. My cousin is psychic - if you draw a card from his deck of cards, he can tell you the name of your card before he looks at it. There's no mechanism for it - it's not a causal thing that scientists could study - he just does it. Same thing when I commune on a deep level with the entire universe in order to realize that my partner truly loves me. I agree that purely spiritual phenomena are outside the realm of causal processes that can be studied by experiments, but I don't agree that they can't be real."
Well, you know, you can stand there all day, shouting all you like about how something is outside the realm of science, but if a picture of the world has this...
...then we're either going to draw an arrow from the top card to the prediction; an arrow from the prediction to the top card (the prediction makes it happen!); or arrows from a third source to both of them (aliens are picking the top card and using telepathy on your cousin... or something; there's no rule you have to label your nodes).
More generally, for me to expect your beliefs to correlate with reality, I have to either think that reality is the cause of your beliefs, expect your beliefs to alter reality, or believe that some third factor is influencing both of them.
This is the more general argument that "To draw an accurate map of a city, you have to open the blinds and look out the window and draw lines on paper corresponding to what you see; sitting in your living-room with the blinds closed, making stuff up, isn't going to work."
Correlation requires causal interaction; and expecting beliefs to be true means expecting the map to correlate with the territory. To open your eyes and look at your shoelaces is to let those shoelaces have a causal effect on your brain - in general, looking at something, gaining information about it, requires letting it causally affect you. Learning about X means letting your brain's state be causally determined by X's state. The first thing that happens is that your shoelace is untied; the next thing that happens is that the shoelace interacts with your brain, via light and eyes and the visual cortex, in a way that makes your brain believe your shoelace is untied.
This is related in spirit to the idea seen earlier on LW that having knowledge materialize from nowhere directly violates the second law of thermodynamics because mutual information counts as thermodynamic negentropy. But the causal form of the proof is much deeper and more general. It applies even in universes like Conway's Game of Life where there's no equivalent of the second law of thermodynamics. It applies even if we're in the Matrix and the aliens can violate physics at will. Even when entropy can go down, you still can't learn about things without being causally connected to them.
The fundamental question of rationality, "What do you think you know and how do you think you know it?", is on its strictest level a request for a causal model of how you think your brain ended up mirroring reality - the causal process which accounts for this supposed correlation.
You might not think that this would be a useful question to ask - that when your brain has an irrational belief, it would automatically have irrational beliefs about process.
But "the human brain is not illogically omniscient", we might say. When our brain undergoes motivated cognition or other fallacies, it often ends up strongly believing in X, without the unconscious rationalization process having been sophisticated enough to also invent a causal story explaining how we know X. "How could you possibly know that, even if it was true?" is a more skeptical form of the same question. If you can successfully stop your brain from rationalizing-on-the-spot, there actually is this useful thing you can sometimes catch yourself in, wherein you go, "Oh, wait, even if I'm in a world where AI does get developed on March 4th, 2029, there's no lawful story which could account for me knowing that in advance - there must've been some other pressure on my brain to produce that belief."
Since it illustrates an important general point, I shall now take a moment to remark on the idea that science is merely one magisterium, and there's other magisteria which can't be subjected to standards of mere evidence, because they are special. That seeing a ghost, or knowing something because God spoke to you in your heart, is an exception to the ordinary laws of epistemology.
That exception would be convenient for the speaker, perhaps. But causality is more general than that; it is not excepted by such hypotheses. "I saw a ghost", "I mysteriously sensed a ghost", "God spoke to me in my heart" - there's no difficulty drawing those causal diagrams.
The methods of science - even sophisticated methods like the conditions for randomizing a trial - aren't just about atoms, or quantum fields.
They're about stuff that makes stuff happen, and happens because of other stuff.
In this world there are well-paid professional marketers, including philosophical and theological marketers, who have thousands of hours of practice convincing customers that their beliefs are beyond the reach of science. But those marketers don't know about causal models. They may know about - know how to lie persuasively relative to - the epistemology used by a Traditional Rationalist, but that's crude by the standards of today's rationality-with-math. Highly Advanced Epistemology hasn't diffused far enough for there to be explicit anti-epistemology against it.
And so we shouldn't expect to find anyone with a background story which would justify evading science's skeptical gaze. As a matter of cognitive science, it seems extremely likely that the human brain natively represents something like causal structure - that this native representation is how your own brain knows that "If the radio says there was an earthquake, it's less likely that your burglar alarm going off implies a burglar." People who want to evade the gaze of science haven't read Judea Pearl's book; they don't know enough about formal causality to not automatically reason this way about things they claim are in separate magisteria. They can say words like "It's not mechanistic", but they don't have the mathematical fluency it would take to deliberately design a system outside Judea Pearl's box.
So in all probability, when somebody says, "I communed holistically and in a purely spiritual fashion with the entire universe - that's how I know my partner loves me, not because of any mechanism", their brain is just representing something like this:
True, false, or meaningless, this belief isn't beyond investigation by standard rationality.
Because causality isn't a word for a special, restricted domain that scientists study. 'Causal process' sounds like an impressive formal word that would be used by people in lab coats with doctorates, but that's not what it means.
'Cause and effect' just means "stuff that makes stuff happen and happens because of other stuff". Any time there's a noun, a verb, and a subject, there's causality. If the universe spoke to you in your heart - then the universe would be making stuff happen inside your heart! All the standard theorems would still apply.
Whatever people try to imagine that science supposedly can't analyze, it just ends up as more "stuff that makes stuff happen and happens because of other stuff".
Mainstream status.
Part of the sequence Highly Advanced Epistemology 101 for Beginners
Next post: "Causal Reference"
Previous post: "Causal Diagrams and Causal Models"