Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

From Philosophy to Math to Engineering

16 Post author: lukeprog 04 November 2013 03:43PM

Cross-posted from the MIRI blog.

For centuries, philosophers wondered how we could learn what causes what. Some argued it was impossible, or possible only via experiment. Others kept hacking away at the problem, clarifying ideas like counterfactual and probability and correlation by making them more precise and coherent.

Then, in the 1990s, a breakthrough: Judea Pearl and others showed that, in principle, we can sometimes infer causal relations from data even without experiment, via the mathematical machinery of probabilistic graphical models.

Next, engineers used this mathematical insight to write software that can, in seconds, infer causal relations from a data set of observations.

Across the centuries, researchers had toiled away, pushing our understanding of causality from philosophy to math to engineering.

From Philosophy to Math to Engineering (small)

And so it is with Friendly AI research. Current progress on each sub-problem of Friendly AI lies somewhere on a spectrum from philosophy to math to engineering.

We began with some fuzzy philosophical ideas of what we want from a Friendly AI (FAI). We want it to be benevolent and powerful enough to eliminate suffering, protect us from natural catastrophes, help us explore the universe, and otherwise make life awesome. We want FAI to allow for moral progress, rather than immediately reshape the galaxy according to whatever our current values happen to be. We want FAI to remain beneficent even as it rewrites its core algorithms to become smarter and smarter. And so on.

Small pieces of this philosophical puzzle have been broken off and turned into math, e.g. Pearlian causal analysis and Solomonoff induction. Pearl's math has since been used to produce causal inference software that can be run on today's computers, whereas engineers have thus far succeeded in implementing (tractable approximations of) Solomonoff induction only for very limited applications.

Toy versions of two pieces of the "stable self-modification" problem were transformed into math problems in de Blanc (2011) and Yudkowsky & Herreshoff (2013), though this was done to enable further insight via formal analysis, not to assert that these small pieces of the philosophical problem had been solved to the level of math.

Thanks to Patrick LaVictoire and other MIRI workshop participants,1 Douglas Hofstadter's FAI-relevant philosophical idea of "superrationality" seems to have been, for the most part, successfully transformed into math, and a bit of the engineering work has also been done.

I say "seems" because, while humans are fairly skilled at turning math into feats of practical engineering, we seem to be much less skilled at turning philosophy into math, without leaving anything out. For example, some very sophisticated thinkers have claimed that "Solomonoff induction solves the problem of inductive inference," or that "Solomonoff has successfully invented a perfect theory of induction." And indeed, it certainly seems like a truly universal induction procedure. However, it turns out that Solomonoff induction doesn't fully solve the problem of inductive inference, for relatively subtle reasons.2

Unfortunately, philosophical mistakes like this could be fatal when humanity builds the first self-improving AGI (Yudkowsky 2008).3 FAI-relevant philosophical work is, as Nick Bostrom says, "philosophy with a deadline."

 

 

1 And before them, Moshe Tennenholtz.

2 Yudkowsky plans to write more about how to improve on Solomonoff induction, later.

3 This is a specific instance of a problem Peter Ludlow described like this: "the technological curve is pulling away from the philosophy curve very rapidly and is about to leave it completely behind."

Comments (42)

Comment author: vallinder 04 November 2013 07:30:02PM 16 points [-]

Patrick Suppes to the left?

Comment author: joaolkf 05 November 2013 04:33:33AM *  0 points [-]

I would buy a modapuccino¹ if we lived in a 100km radius!

  1. Modafinil's version of a cappuccino
Comment author: vallinder 05 November 2013 11:45:13AM 0 points [-]

Well, I hope you're in Oxford soon again, João! :)

Comment author: joaolkf 05 November 2013 07:17:44PM 0 points [-]

aah, it's you! :) We might indeed go for a fika sometime in the near future then. I wonder if you were following the Swedish-transhumanism-FHI squabble on my last post.

Comment author: lukeprog 05 November 2013 04:28:40AM 0 points [-]

Yes! Less Wrong, please give vallinder extra karma, for identifying the more ambiguous cartoon dude. :)

Comment author: komponisto 05 November 2013 08:22:17AM 0 points [-]
Comment author: shminux 05 November 2013 07:09:46AM *  -2 points [-]

From his wikipedia page:

has made significant contributions to philosophy of science, the theory of measurement, the foundations of quantum mechanics

Sounds like either self-advertisement or a hero-worship, depending on who wrote it.

Comment author: pragmatist 07 November 2013 01:43:42PM *  2 points [-]

I'm confused. Why does it sound like that to you? Saying someone has made "significant contributions" doesn't sound excessively adulatory, especially if you're talking about someone who is widely regarded as having made significant contributions (as evidenced by, say, his National Medal of Science).

Comment author: shminux 07 November 2013 05:41:54PM 1 point [-]

First, I never heard of Patrick Suppes in any of my physics studies. Or at all, until he was mentioned here. So this puts a cap on the significance of his contributions to the "foundations of physics". Second, the medal is "National Medal of Science for Behavioral and Social Science", not anything physics-related (unless you consider "subjective probability" physics). Now, my attitude toward "philosophy of science" roughly matches that of Feynman, so I am not going to touch that one. What's left is "the theory of measurement", which am not in a position to evaluate, except to say that, from the list of relevant publications it seems like he worked in the decision-theoretic area, rather than any physical measurement. Now, maybe he did make "significant contributions" to decision theory, I cannot tell, except that this forum does not mention him much while discussing decision-theoretic issues.

Not to put him down, he did a lot of interesting work and is likely in the top 0.1% of physics PhDs by various metrics, just not as "foundational" as the wiki article presents him.

Comment author: vallinder 05 November 2013 11:58:59AM 1 point [-]

I'm not familiar with his writings on the foundations of quantum mechanics, but in addition to his work on causality, the three volumes on measurement he co-authored have also been hugely influential. His intellectual autobiography (pdf) might be worth a look.

Comment author: joaolkf 05 November 2013 04:28:06AM 0 points [-]

I think you might have finally got it. I hope so at least, this google image madness has to stop.

Comment author: passive_fist 05 November 2013 12:14:17AM 1 point [-]

The 'robust cooperation' paper is very interesting, thanks! Is there an outline of it for those of us who would like to learn about it but do not have much time to spare?

Comment author: Adele_L 05 November 2013 02:37:27AM 3 points [-]

Well you could read the associated post.

If this isn't what you are looking for, and you clarify what you are looking for, I could probably write some sort of outline.

Comment author: joaolkf 04 November 2013 06:15:05PM 0 points [-]

http://leiterreports.typepad.com/blog/2013/10/progress-in-philosophy-revisited.html

Wait a minute, there's a senior philosopher, independently from MIRI/FHI, going around giving talks about how we should invest effort and money on philosophically addressing the most important problems? Why aren't we funding this? He should be 'lobbying' universities' deans and whatnot.

Also, you just mistakenly pictured Russell while attempting to draw another guy. Left dude IS Russell.

Comment author: lukeprog 05 November 2013 04:29:56AM 1 point [-]

The source material sent to the odesk cartoonist was a photo of Suppes, believe it or not. :)

Comment author: joaolkf 05 November 2013 04:37:01AM 0 points [-]
Comment author: IlyaShpitser 04 November 2013 04:15:19PM 0 points [-]

Out of curiosity, who are the left and right dudes supposed to be?

Comment author: gjm 04 November 2013 05:52:26PM *  0 points [-]

Left-hand dude looks a bit like Immanuel Kant. (I'd have thought David Hume was a more likely choice, but he didn't look much like that.)

[EDITED to clarify.]

Comment author: Jayson_Virissimo 04 November 2013 06:13:50PM 0 points [-]

Left-hand dude looks a bit like Immanuel Kant.

The attire is much too modern to be Kant.

Comment author: gjm 04 November 2013 06:40:07PM 0 points [-]

That's very true.

Comment author: lukeprog 04 November 2013 04:31:36PM *  0 points [-]

Karma points to whoever identifies them correctly!

The guy on the right is more accurately represented than the guy on the left, but the guy on the left may still be guessable based on the subject matter. Hint: the guy on the left is a philosopher known, among other things, for his post-1950 contributions to theory of causality.

(The guy in the middle is Judea Pearl, Ilya's advisor at UCLA.)

Update: dougclow correctly identified the man on the right, and vallinder correctly identified the man on the left.

Comment author: dougclow 04 November 2013 05:00:28PM 16 points [-]

Guy on the right is Markus Kalisch.

Not sure about the one on the left - outside chance it's Bertrand Russell but probably not.

Comment author: lukeprog 04 November 2013 05:06:29PM 0 points [-]

Give the man some points!

Guy on the right is indeed Markus Kalisch.

The guy on the left is not Bertrand Russell.

Comment author: Discredited 05 November 2013 03:15:32AM *  0 points [-]

The man on the left is Hans Reichenbach.

Comment author: lukeprog 05 November 2013 04:21:49AM 1 point [-]

Nope!

Comment author: Discredited 05 November 2013 05:09:10AM 0 points [-]

Richard Jeffrey!

Comment author: lukeprog 05 November 2013 05:36:11AM 0 points [-]

Nope, Suppes.

Comment author: gjm 05 November 2013 03:30:22AM 0 points [-]

That was one of my first guesses, but it doesn't seem to look much like him.

Comment author: spqr0a1 04 November 2013 10:32:07PM 0 points [-]

On the left is Willard Quine.

Comment author: lukeprog 05 November 2013 04:21:58AM 0 points [-]

Nope!

Comment author: gjm 04 November 2013 11:19:06PM 0 points [-]

How sure are you? It seems plausible that Quine looked like that at one time, but none of the pictures of him I can find look all that similar. And while Quine did important work on many many things, I wouldn't put causality high on the list of things he's known for working on.

Still, I've so far failed to find anyone notable in the field who looks much more like that image...

Comment author: Jayson_Virissimo 04 November 2013 06:17:41PM 0 points [-]

Is the guy on the left Sewall Wright?

Comment author: Jayson_Virissimo 04 November 2013 06:38:38PM 0 points [-]

Whoops, you said post-1950s.

Comment author: IlyaShpitser 04 November 2013 07:05:16PM 0 points [-]

Also Sewall Wright was no philosopher -- he was a biologist. Faced with a problem on the left, he basically correctly formalized and solved it for the linear/Gaussian case. He should be in the middle with Judea and on the right with engineers!

Comment author: TsviBT 04 November 2013 05:43:13PM 0 points [-]

Guy on the left is Max Born?

Comment author: lukeprog 04 November 2013 05:45:05PM 0 points [-]

Nope.

Comment author: joaolkf 04 November 2013 05:03:14PM 0 points [-]

Left is Russell. Haircut.

Comment author: lukeprog 04 November 2013 05:08:54PM 0 points [-]

A fine guess, but no.

Comment author: joaolkf 04 November 2013 05:26:51PM 1 point [-]

Meh. Right is Markus Kalisch?

Comment author: lukeprog 04 November 2013 05:30:08PM 0 points [-]

dougclow narrowly beat you to it.

Comment author: joaolkf 04 November 2013 05:06:47PM 0 points [-]

Mirrored Russell? His hair divides on the right, not left. But the subject matter is Russellian.

Comment author: fubarobfusco 05 November 2013 04:07:55AM 0 points [-]

The one on the left looks a bit like Bertrand Russell.