Eliezer_Yudkowsky comments on Less Wrong Rationality and Mainstream Philosophy - Less Wrong

106 Post author: lukeprog 20 March 2011 08:28PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (328)

You are viewing a single comment's thread. Show more comments above.

Comment author: lukeprog 21 March 2011 06:10:42PM *  53 points [-]

Eliezer,

When I wrote the post I didn't know that what you meant by "reductionist-grade naturalistic cognitive philosophy" was only the very narrow thing of dissolving philosophical problems to cognitive algorithms. After all, most of the useful philosophy you've done on Less Wrong is not specifically related to that very particular thing... which again supports my point that mainstream philosophy has more to offer than dissolution-to-algorithm. (Unless you think most of your philosophical writing on Less Wrong is useless.)

Also, I don't disagree with your decision not to cover means and ends in CEV.

Anyway. Here are some useful contributions of mainstream philosophy:

  • Quine's naturalized epistemology. Epistemology is a branch of cognitive science: that's where recursive justification hits bottom, in the lens that sees its flaws.
  • Tarski on language and truth. One of Tarski's papers on truth recently ranked as the 4th most important philosophy paper of the 20th century by a survey of philosophers. Philosophers have much developed Tarski's account since then, of course.
  • Chalmers' formalization of Good's intelligence explosion argument. Good's 1965 paper was important, but it presented no systematic argument; only hand-waving. Chalmers breaks down Good's argument into parts and examines the plausibility of each part in turn, considers the plausibility of various defeaters and possible paths, and makes a more organized and compelling case for Good's intelligence explosion than anybody at SIAI has.
  • Dennett on belief in belief. Used regularly on Less Wrong.
  • Bratman on intention. Bratman's 1987 book on intention has been a major inspiration to AI researchers working on belief-desire-intention models of intelligent behavior. See, for example, pages 60-61 and 1041 of AIMA (3rd ed.).
  • Functionalism and multiple realizability. The philosophy of mind most natural to AI was introduced and developed by Putnam and Lewis in the 1960s, and more recently by Dennett.
  • Explaining the cognitive processes that generate our intuitions. Both Shafir (1998) and Talbot (2009) summarize and discuss as much as cognitive scientists know about the cognitive mechanisms that produce our intuitions, and use that data to explore which few intuitions might be trusted and which ones cannot - a conclusion that of course dissolves many philosophical problems generated from conflicts between intuitions. (This is the post I'm drafting, BTW.) Talbot describes the project of his philosophy dissertation for USC this way: "...where psychological research indicates that certain intuitions are likely to be inaccurate, or that whole categories of intuitions are not good evidence, this will overall benefit philosophy. This has the potential to resolve some problems due to conflicting intuitions, since some of the conflicting intuitions may be shown to be unreliable and not to be taken seriously; it also has the potential to free some domains of philosophy from the burden of having to conform to our intuitions, a burden that has been too heavy to bear in many cases..." Sound familiar?
  • Pearl on causality. You acknowledge the breakthrough. While you're right that this is mostly a case of an AI researcher coming in from the outside to solve philosophical problems, Pearl did indeed make use of the existing research in mainstream philosophy (and AI, and statistics) in his book on causality.
  • Drescher's Good and Real. You've praised this book as well, which is the result of Drescher's studies under Dan Dennett at Tufts. And the final chapter is a formal defense of something like Kant's categorical imperative.
  • Dennett's "intentional stance." A useful concept in many contexts, for example here.
  • Bostrom on anthropic reasoning. And global catastrophic risks. And Pascal's mugging. And the doomsday argument. And the simulation argument.
  • Ord on risks with low probabilities and high stakes. Here.
  • Deontic logic. The logic of actions that are permissible, forbidden, obligatory, etc. Not your approach to FAI, but will be useful in constraining the behavior of partially autonomous machines prior to superintelligence, for example in the world's first battlefield robots.
  • Reflective equilibrium. Reflective equilibrium is used in CEV. It was first articulated by Goodman (1965), then by Rawls (1971), and in more detail by Daniels (1996). See also the more computational discussion in Thagard (1988), ch. 7.
  • Experimental philosophy on the biases that infect our moral judgments. Experimental philosophers are now doing Kahneman & Tversky -ish work specific to biases that infect our moral judgments. Knobe, Nichols, Haidt, etc. See an overview in Experiments in Ethics.
  • Greene's work on moral judgment. Joshua Greene is a philosopher and neuroscientist at Harvard whose work using brain scanners and trolley problems (since 2001) is quite literally decoding the algorithms we use to arrive at moral judgments, and helping to dissolve the debate between deontologists and utilitarians (in his view, in favor of utilitarianism).
  • Dennett's Freedom Evolves. The entire book is devoted to explaining the evolutionary processes that produced the cognitive algorithms that produce the experience of free will and the actual kind of free will we do have.
  • Quinean naturalists showing intuitionist philosophers that they are full of shit. See for example, Schwitzgebel and Cushman demonstrating experimentally that moral philosophers have no special expertise in avoiding known biases. This is the kind of thing that brings people around to accepting those very basic starting points of Quinean naturalism as a first step toward doing useful work in philosophy.
  • Bishop & Trout on ameliorative psychology. Much of Less Wrong's writing is about how to use our awareness of cognitive biases to make better decisions and have a higher proportion of beliefs that are true. That is the exact subject of Bishop & Trout (2004), which they call "ameliorative psychology." The book reads like a long sequence of Less Wrong posts, and was the main source of my post on statistical prediction rules, which many people found valuable. And it came about two years before the first Eliezer post on Overcoming Bias. If you think that isn't useful stuff coming from mainstream philosophy, then you're saying a huge chunk of Less Wrong isn't useful.
  • Talbot on intuitionism about consciousness. Talbot (here) argues that intuitionist arguments about consciousness are illegitimate because of the cognitive process that produces them: "Recently, a number of philosophers have turned to folk intuitions about mental states for data about whether or not humans have qualia or phenomenal consciousness. [But] this is inappropriate. Folk judgments studied by these researchers are mostly likely generated by a certain cognitive system - System One - that will ignore qualia when making these judgments, even if qualia exist."
  • "The mechanism behind Gettier intuitions." This upcoming project of the Boulder philosophy department aims to unravel a central (misguided) topic of 20th century epistemology by examining the cognitive mechanisms that produce the debate. Dissolution to algorithm yet again. They have other similar projects ongoing, too.
  • Computational meta-ethics. I don't know if Lokhorst's paper in particular is useful to you, but I suspect that kind of thing will be, and Lokhorst's paper is only the beginning. Lokhorst is trying to implement a meta-ethical system computationally, and then actually testing what the results are.

Of course that's far from all there is, but it's a start.

...also, you occasionally stumble across some neato quotes, like Dennett saying "AI makes philosophy honest." :)

Note that useful insights come from unexpected places. Rawls was not a Quinean naturalist, but his concept of reflective equilibrium plays a central role in your plan for Friendly AI to save the world.

P.S. Predicate logic was removed from the original list for these reasons.

Comment author: Eliezer_Yudkowsky 21 March 2011 07:55:02PM 9 points [-]

When I wrote the post I didn't know that what you meant by "reductionist-grade naturalistic cognitive philosophy" was only the very narrow thing of dissolving philosophical problems to cognitive algorithms.

No, it's more than that, but only things of that level are useful philosophy. Other things are not philosophy or more like background intros.

Amy just arrived and I've got to start book-writing, but I'll take one example from this list, the first one, so that I'm not picking and choosing; later if I've got a moment I'll do some others, in the order listed.

  • Predicate logic.

Funny you should mention that.

There is this incredibly toxic view of predicate logic that I first encountered in Good Old-Fashioned AI. And then this entirely different, highly useful and precise view of the uses and bounds of logic that I encountered when I started studying mathematical logic and learned about things like model theory.

Now considering that philosophers of the sort I inveighed against in "against modal logic" seem to talk and think like the GOFAI people and not like the model-theoretic people, I'm guessing that the GOFAI people made the terrible, horrible, no good, very bad mistake of getting their views of logic from the descendants of Bertrand Russell who still called themselves "philosophers" instead of those descendants who considered themselves part of the thriving edifice of mathematics.

Anyway. If you and I agree that philosophy is an extremely sick field, that there is no standardized repository of the good stuff, that it would be a desperate and terrible mistake for anyone to start their life studying philosophy before they had learned a lot of cognitive science and math and AI algorithms and plain old material science as explained by non-philosophers, and that it's not worth my time to read through philosophy to pick out the good stuff even if there are a few small nuggets of goodness or competent people buried here and there, then I'm not sure we disagree on much - except this post sort of did seem to suggest that people ought to run out and read philosophy-qua-philosophy as written by professional philosophers, rather than this being a terrible mistake.

Will try to get to some of the other items, in order, later.

Comment author: lukeprog 14 May 2011 03:14:06AM 11 points [-]

You may enjoy the following exchange between two philosophers and one mathematician.

Bertrand Russell, speaking of Godel's incompleteness theorem, wrote:

It made me glad that I was no longer working at mathematical logic. If a given set of axioms leads to a contradiction, it is clear that at least one of the axioms must be false.

Wittgenstein dismissed the theorem as trickery:

Mathematics cannot be incomplete; any more than a sense can be incomplete. Whatever I can understand, I must completely understand.

Godel replied:

Russell evidently misinterprets my result; however, he does so in a very interesting manner... In contradistinction Wittgenstein... advances a completely trivial and uninteresting misinterpretation.

According to Gleick (in The Information), the only person who understood Godel's theorem when Godel first presented it was another mathematician, Neumann Janos, who moved to the USA and began presenting it wherever he went, by then calling himself John von Neumann.

The soundtrack for Godel's incompleteness theorem should be, I think, the last couple minutes of 'Ludus' from Tabula Rasa by Arvo Part.

Comment author: Wei_Dai 14 May 2011 08:22:07AM *  12 points [-]

I've been wondering why von Neumann didn't do much work in the foundations of mathematics. (It seems like something he should have been very interested in.) Your comment made me do some searching. It turns out:

John von Neumann was a vain and brilliant man, well used to putting his stamp on a mathematical subject by sheer force of intellect. He had devoted considerable effort to the problem of the consistency of arithmetic, and in his presentation at the Konigsberg symposium, had even come forward as an advocate for Hilbert's program. Seeing at once the profound implications of Godel's achievement, he had taken it one step further—proving the unprovability of consistency, only to find that Godel had anticipated him. That was enough. Although full of admiration for Godel—he'd even lectured on his work—von Neumann vowed never to have anything more to do with logic. He is said to have boasted that after Godel, he simply never read another paper on logic. Logic had humiliated him, and von Neumann was not used to being humiliated. Even so, the vow proved impossible to keep, for von Neumann's need for powerful computational machinery eventually forced him to return to logic.

ETA: Am I the only one who fantasizes about cloning a few dozen individuals from von Neumann's DNA, teaching them rationality, and setting them to work on FAI? There must be some Everett branches where that is being done, right?

Comment author: lukeprog 14 May 2011 08:35:19AM 2 points [-]

We'd need to inoculate the clones against vanity, it appears.

Interesting story. Thanks for sharing your findings.

Comment author: wedrifid 14 May 2011 07:02:26AM -1 points [-]

Russell evidently misinterprets my result; however, he does so in a very interesting manner... In contradistinction Wittgenstein... advances a completely trivial and uninteresting misinterpretation.

Well spoken! :)

Comment author: Oscar_Cunningham 21 March 2011 08:12:34PM 10 points [-]

Of course, since this is a community blog, we can have it both ways. Those of us interested in philosophy can go out and read (and/or write) lots of it, and we'll chuck the good stuff this way. No need for anyone to miss out.

Comment author: lukeprog 21 March 2011 09:15:49PM 5 points [-]

Exactly. Like I did with my statistical prediction rules post.

Comment author: lukeprog 21 March 2011 08:04:03PM *  6 points [-]

Anyway. If you and I agree...

Yeah, we don't disagree much on all those points.

I didn't say in my original post that people should run out and start reading mainstream philosophy. If that's what people got from it, then I'll add some clarifications to my original post.

Instead, I said that mainstream philosophy has some useful things to offer, and shouldn't be ignored. Which I think you agree with if you've benefited from the work of Bostrom and Dennett (including, via Drescher) and so on. But maybe you still disagree with it, for reasons that are forthcoming in your response to my other examples of mainstream philosophy contributions useful to Less Wrong.

But yeah, don't let me keep you from your book!

As for predicate logic, I'll have to take your word on that. I'll 'downgrade it' in my list above.

Comment author: TheOtherDave 21 March 2011 08:15:37PM 11 points [-]

If that's what people got from it, then I'll add some clarifications to my original.

FWIW, what I got from your original post was not "LW readers should all go out and start reading mainstream philosophy," but rather "LW is part of a mainstream philosophical lineage, whether its members want to acknowledge that or not."

Comment author: lukeprog 21 March 2011 08:22:15PM 2 points [-]

Thanks for sharing. That too. :)

Comment author: Eliezer_Yudkowsky 22 March 2011 12:08:11AM 1 point [-]

I'm part of Roger Bacon's lineage too, and not ashamed of it either, but time passes and things improve and then there's not much point in looking back.

Comment author: lukeprog 22 March 2011 12:21:57AM *  15 points [-]

Meh. Historical context can help put things in perspective. You've done that plenty of times in your own posts on Less Wrong. Again, you seem to be holding my post to a different standard of usefulness than your own posts. But like I said, I don't recommend anybody actually read Quine.

Comment author: [deleted] 02 April 2015 05:54:59PM 1 point [-]

Oftentimes you simply can't understand what some theorem or experiment was for without at least knowing about its historical context. Take something as basic as calculus: if you've never heard the slightest thing about classical mechanics, what possible meaning could a derivative, integral, or differential equation have to you?

Comment author: TheAncientGeek 02 April 2015 07:25:10PM 0 points [-]

Does human nature improve, too?

Comment author: ChristianKl 02 April 2015 07:38:45PM 0 points [-]

What's "human nature"?

Comment author: TheAncientGeek 12 April 2015 06:50:27PM 0 points [-]

Something that probably hasn't changed much over the history of philosophy.

Comment author: Perplexed 21 March 2011 11:24:09PM *  8 points [-]

There is this incredibly toxic view of predicate logic that I first encountered in Good Old-Fashioned AI.

I'd be curious to know what that "toxic view" was. My GOFAI academic advisor back in grad school swore by predicate logic. The only argument against that I ever heard was that proving or disproving something is undecidable (in theory) and frequently intractible (in practice).

And then this entirely different, highly useful and precise view of the uses and bounds of logic that I encountered when I started studying mathematical logic and learned about things like model theory.

Model theory as opposed to proof theory? What is it you think is great about model theory?

Now considering that philosophers of the sort I inveighed against in "against modal logic" seem to talk and think like the GOFAI people and not like the model-theoretic people, I'm guessing that the GOFAI people made the terrible, horrible, no good, very bad mistake of getting their views of logic from the descendants of Bertrand Russell who still called themselves "philosophers" instead of those descendants who considered themselves part of the thriving edifice of mathematics.

I have no idea what you are saying here. That "Against Modal Logic" posting, and some of your commentary following it strike me as one of your most bizarre and incomprehensible pieces of writing at OB. Looking at the karma and comments suggests that I am not alone in this assessment.

Somehow, you have picked up a very strange notion of what modal logic is all about. The whole field of hardware and software verification is based on modal logics. Modal logics largely solve the undecidability and intractability problems the bedeviled GOFAI approaches to these problems using predicate logic. Temporal logics are modal. Epistemic and game-theoretic logics are modal.

Or maybe it is just the philosophical approaches to modal logic that offended you. The classical modal logic of necessity and possibility. The puzzles over the Barcan formulas when you try to combine modality and quantification. Or maybe something bizarre involving zombies or Goedel/Anselm ontological proofs.

Whatever it was that poisoned your mind against modal logic, I hope it isn't contagious. Modal logic is something that everyone should be exposed to, if they are exposed to logic at all. A classic introductory text: Robert Goldblatt: Logics of Time and Computation (pdf) is now available free online. I just got the current standard text from the library. It - Blackburn et al.: Modal Logic (textbook) - is also very good. And the standard reference work - Blackburn et al.: Handbook of Modal Logic - is outstanding (and available for less than $150 as Borders continues to go out of business :)

Comment author: lukeprog 21 March 2011 11:35:45PM 7 points [-]

Reading Plantinga could poison almost anybody's opinion of modal logic. :)

Comment author: Perplexed 21 March 2011 11:50:37PM 3 points [-]

That is entirely possible. A five star review at the Amazon link you provided calls this "The classic work on the metaphysics of modality". Another review there says:

Plantinga's Nature of Necessity is a philosophical masterpiece. Although there are a number of good books in analytic philosophy dealing with modality (the concepts of necessity and possibility), this one is of sufficient clarity and breadth that even non-philosophers will benefit from it. Modal logic may seem like a fairly arcane subject to outsiders, but this book exhibits both its intrinsic interest and its general importance.

Yet among the literally thousands of references in the three books I linked, Platinga is not even mentioned. A fact which pretty much demonstrates that modal logic has left mainstream philosophy behind. Modal logic (in the sense I am promoting) is a branch of logic, not a branch of metaphysics.

Comment author: PhilGoetz 02 April 2011 03:04:02PM 3 points [-]

There is this incredibly toxic view of predicate logic that I first encountered in Good Old-Fashioned AI. And then this entirely different, highly useful and precise view of the uses and bounds of logic that I encountered when I started studying mathematical logic and learned about things like model theory.

I'd very much like to see a post explaining that.

Comment author: lukeprog 22 March 2011 01:32:08PM *  3 points [-]

it's more than that, but only things of that level are useful philosophy. Other things are not philosophy or more like background intros.

I'm not sure what "of that level" (of dissolving-to-algorithm) means, but I think I've demonstrated that quite a lot of useful stuff comes from mainstream philosophy, and indeed that a lot of mainstream philosophy is already being used by yourself and Less Wrong.