Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Eliezer_Yudkowsky comments on Rationality Quotes: April 2011 - Less Wrong

6 Post author: benelliott 04 April 2011 09:55AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (384)

You are viewing a single comment's thread. Show more comments above.

Comment author: Nominull 04 April 2011 01:35:51PM *  43 points [-]

On the plus side, bad things happening to you does not mean you are a bad person. On the minus side, bad things will happen to you even if you are a good person. In the end you are just another victim of the motivationless malice of directed acyclic causal graphs.

-Nobilis RPG 3rd edition

Comment author: Eliezer_Yudkowsky 04 April 2011 04:22:09PM 7 points [-]

...that was written by a Less Wrong reader. Or if not, someone who independently reinvented things to well past the point where I want to talk to them. Do you know the author?

Comment author: JoshuaZ 04 April 2011 04:29:21PM 9 points [-]

The author of most of the Nobilis work is Jenna K. Moran. I'm unsure if this remark is independent of LW or not. The Third Edition (where that quote is from) was published this year, so it is possible that LW influenced it.

Comment author: HonoreDB 04 April 2011 05:36:14PM *  4 points [-]

Heh, I clicked the link to see when she took over Nobilis from Rebecca Borgstrom, only to find that she took over more than that from her.

Edit: Also, serious memetic hazard warning with regard to her fiction blog, which is linked from the article.

Comment author: novalis 04 April 2011 08:57:02PM *  22 points [-]

I'm not sure it's a memetic hazard, but this post is one of the most Hofstadterian things outside of Hofstadter

Until this moment, I had always assumed that Eliezer had read 100% of all fiction.

Comment author: Sniffnoy 05 April 2011 11:28:59PM 8 points [-]

Or just someone else who read Pearl, no?

Comment author: Tyrrell_McAllister 06 April 2011 03:32:43AM *  5 points [-]

...that was written by a Less Wrong reader. Or if not, someone who independently reinvented things to well past the point where I want to talk to them. Do you know the author?

Hasn't using DAGs to talk about causality long been a staple of the philosophy and computer science of causation? The logical positivist philosopher Hans Reichenbach used directed acyclic graphs to depict causal relationships between events in his book The Direction of Time (1956). (See, e.g., p. 37.)

A little searching online also turned up this 1977 article in Proc Annu Symp Comput Appl Med Care. From p. 72:

When a set of cause and effect relationships between states is specified, the resulting structure is a network, or directed acyclic graph of states.

That article came out around the time of Pearl's first papers, and it doesn't cite him. Had his ideas already reached that level of saturation?

ETA: I've looked a little more closely at the 1977 paper, which is entitled "Problems in the Design of Knowledge Bases for Medical Consultation". It appears to completely lack the idea of performing surgery on the DAGs, though I may have missed something. Here is a longer quote from the paper (p. 72):

Many states may occur simultaneously in any disease process. A state thus defined may be viewed as a qualitative restriction on a state variable as used in control systems theory. It does not correspond to one of the mutually exclusive states that could be used to describe a probabilistic system.

[...]

When a set of cause and effect relationships between states is specified, the resulting structure is a network, or directed acyclic graph of states.

The mappings between nodes n_i of the causal net are of n_i -- a_{ij} --> n_j where a_{ij} is the strength of causation (interpreted in terms of its frequency of occurrence) and n_i and n_j are states which are summarized by English language statements. This rule is interpreted as: state n_i causes state n_j, independent of other events, with frequency a_{ij}. Starting states are also assigned a frequency measure indicating a prior or starting frequency. The levels of causation are represented by numerical values, fractions between zero and one, which correspond to qualitative ranges such as: sometimes, often, usually, or always.

So, when it comes to demystifying causation, there is still a long distance from merely using DAGs to using DAGs in the particularly insightful way that Pearl does.

Comment author: IlyaShpitser 11 April 2011 01:35:51AM *  8 points [-]

Hi, you might want to consider this paper:

http://www.ssc.wisc.edu/soc/class/soc952/Wright/Wright_The%20Method%20of%20Path%20Coefficients.pdf

This paper is remarkable not only because it correctly formalizes causation in linear models using DAGs, but also that it gives a method for connecting causal and observational quantities in a way that's still in use today. (The method itself was proposed in 1923, I believe). Edit: apparently in 1920-21, with earliest known reference apparently dating back to 1918.

Using DAGs for causality certainly predates Pearl. Identifying "randomization on X" with "dividing by P(x | pa(x))" might be implicit in fairly old papers also. Again, this idea predates Pearl.

There's always more to the story than one insightful book.

Comment author: cousin_it 11 April 2011 09:22:14AM *  4 points [-]

Good find, thanks. The handwritten equations are especially nice.

Ilya, it looks you're the perfect person to write an introductory LW post about causal graphs. We don't have any good intro to the topic showing why it is important and non-obvious (e.g. the smoking/tar/cancer example). I'm willing to read drafts, but given your credentials I think it's not necessary :-)

Comment author: Eliezer_Yudkowsky 07 April 2011 06:03:06AM 4 points [-]

The point is that it's not commonly internalized to the point where someone will correctly use DAG as a synonym for "universe".

Comment author: wedrifid 10 April 2011 08:36:49AM 4 points [-]

The point is that it's not commonly internalized to the point where someone will correctly use DAG as a synonym for "universe".

Synonym? Not just 'capable of being used to perfectly represent', but an actual literal synonym? That's a remarkable claim. I'm not saying I outright don't believe it but it is something I would want to see explained in detail first.

Would reading Pearl (competently) be sufficient to make someone use the term DAG correctly in that sense?

Comment author: Tyrrell_McAllister 07 April 2011 05:41:52PM *  3 points [-]

The point is that it's not commonly internalized to the point where someone will correctly use DAG as a synonym for "universe".

All that I see in the quote is that the DAG is taken to determine what happens to you in some unanalyzed sense. You often hear similar statements saying that the cold equations of physics determine your fate, but the speaker is not necessarily thinking of "equations of physics" as synonymous with "universe".

Comment author: thomblake 07 April 2011 08:38:54PM 3 points [-]

Seriously, she seems pretty awesome. link to Johns Hopkins profile

Comment author: David_Gerard 04 April 2011 04:35:35PM *  2 points [-]

Or if not, someone who independently reinvented things to well past the point where I want to talk to them.

The memes are getting out there! (Hopefully.)

Comment author: Larks 05 April 2011 01:19:13PM 7 points [-]

No, hopefully they were re-discovered. We can improve our publicity skills, but we can't make ideas easier to independantly re-invent.

Comment author: Vaniver 05 April 2011 05:00:02PM 2 points [-]

No, hopefully they were re-discovered. We can improve our publicity skills, but we can't make ideas easier to independantly re-invent.

Really? If meme Z is the result of meme X and Y colliding, then it seems like spreading X and Y makes it easier to independently re-invent Z.

Comment author: Larks 05 April 2011 05:11:14PM 1 point [-]

Yes - by 'independently' I mean 'unaffected by any publicity work we might do'.

Comment author: David_Gerard 05 April 2011 04:53:18PM *  -1 points [-]

I think them surviving as spreading memes is pretty good, if the information is transmitted without important errors creeping in. Though yes, reinventability is good (and implies the successful spread of prerequisite memes).

Comment author: Larks 05 April 2011 05:10:34PM 4 points [-]

Oh yeah, both are good, but like good evidential decision theorists we should hope for re-invention.

Comment author: spriteless 08 April 2011 10:36:47PM 1 point [-]