Note the way I speak with John Baez in the following interview, done months before the present post:
http://johncarlosbaez.wordpress.com/2011/03/25/this-weeks-finds-week-313/
In terms of what I would advocate programming a very powerful AI to actually do, the keywords are “mature folk morality” and “reflective equilibrium”...
In terms of Google keywords, my brand of metaethics is closest to analytic descriptivism or moral functionalism...
I was happy to try and phrase this interview as if it actually had something to do with philosophy.
Although I actually invented the relevant positions myself, on the fly when FAI theory needed it, then Googled around to find the philosophical nearest neighbor.
The fact that you are skeptical about this, and suspect I suppose that I accidentally picked up some analytic descriptivism or mature folk morality elsewhere and then forgot I'd read about it, even though I hadn't gone anywhere remotely near that field of philosophy until I wanted to try speaking their language, well, that strikes at the heart of why all this praise of "mainstream" philosophy strikes me the wrong way. Because the versions of "mature folk morality" and &qu...
With this comment, I think our disagreement is resolved, at least to my satisfaction.
We agree that philosophy can be useful, and that sometimes it's desirable to speak the common language. I agree that sometimes it is easier to reinvent the wheel, but sometimes it's not.
As for whether Less Wrong is a branch of mainstream philosophy, I'm not much interested to argue about that. There are many basic assumptions shared by Quinean philosophy and Yudkowskian philosophy in opposition to most philosophers, even down to some very specific ideas like naturalized epistemology that to my knowledge had not been articulated very well until Quine. And both Yudkowskian philosophy and Quinean naturalism spend an awful lot of time dissolving philosophical debates into cognitive algorithms and challenging intuitionist thinking - so far, those have been the main foci of experimental philosophy, which is very Quinean, and was mostly founded by one of Quine's students, Stephen Stich. Those are the reasons I presented Yudkowskian philosophy as part of the broadly Quinean movement in philosophy.
On the other hand, I'm happy to take your word for it that you came up with most of this stuff on your own, and...
The community definitely needs to work on this whole "virtue of scholarship" thing.
It's not Quinean naturalism. It's logical empiricism with a computational twist. I don't suggest that everyone go out and read Carnap, though. One way that philosophy makes progress is when people work in relative isolation, figuring out the consequences of assumptions rather than arguing about them. The isolation usually leads to mistakes and reinventions, but it also leads to new ideas. Premature engagement can minimize all three.
Philosophy quote of the day:
I am prepared to go so far as to say that within a few years, if there remain any philosophers who are not familiar with some of the main developments in artificial intelligence, it will be fair to accuse them of professional incompetence, and that to teach courses in philosophy of mind, epistemology, aesthetics, philosophy of science, philosophy of language, ethics, metaphysics, and other main areas of philosophy, without discussing the relevant aspects of artificial intelligence will be as irresponsible as giving a degree course in physics which includes no quantum theory.
Aaron Sloman (1978)
According to the link:
Aaron Sloman is a philosopher and researcher on artificial intelligence and cognitive science.
So, we have a spectacular mis-estimation of the time frame - claiming 33 years ago that AI would be seen as important "within a few years". That is off by one order of magnitude (and still counting!) Do we blame his confusion on the fact that he is a philosopher, or was the over-optimism a symptom of his activity as an AI researcher? :)
ETA:
as irresponsible as giving a degree course in physics which includes no quantum theory.
I'm not sure I like the analogy. QM is foundational for physics, while AI merely shares some (as yet unknown) foundation with all those mind-oriented branches of philosophy. A better analogy might be "giving a degree course in biology which includes no exobiology".
Hmmm. I'm reasonably confident that biology degree programs will not include more than a paragraph on exobiology until we have an actual example of exobiology to talk about. So what is the argument for doing otherwise with regard to AI in philosophy?
Oh, yeah. I remember. Philosophers, unlike biologists, have never shied away from investigating things that are not known to exist.
Many mainstream philosophers have been defending Less Wrong-ian positions for decades before Overcoming Bias or Less Wrong existed.
When I read posts on Overcoming Bias (and sometimes also LW) discussing various human frailties and biases, especially those related to status and signaling, what often pops into my mind are observations by Friedrich Nietzsche. I've found that many of them represent typical OB insights, though expressed in a more poetic, caustic, and disorganized way. Now of course, there's a whole lot of nonsense in Nietzsche, and a frightful amount of nonsense in the subsequent philosophy inspired by him, but his insight about these matters is often first-class.
Also, how about William James and pragmatism? I read Pragmatism recently, and had been meaning to post about the many bits that sound like they could've been cut straight from the sequences -- IIRC, there was some actual discussion of making beliefs "pay" -- in precisely the same manner as the sequences speak of beliefs paying rent.
Yup.
Quinean naturalism, and especially Quine's naturalized epistemology, are merely the "fullest" accounts of Less Wrong-ian philosophy to be found in the mainstream literature. Of course particular bits come from earlier traditions.
Parts of pragmatism (Peirce & Dewey) and pre-Quinean naturalism (Sellars & Dewey and even Hume) are certainly endorsed by much of the Less Wrong community. As far as I can tell, Eliezer's theory of truth is straight-up Peircian pragmatism.
My theory of truth is explicitly Tarskian. I'm explicitly influenced by Korzybski on language and by Peirce on "making beliefs pay rent", but I do think there are meaningful and true beliefs such that we cannot experientally distinguish between them and mutually exclusive alternatives, i.e., a photon going on existing after it passes over the horizon of the expanding universe as opposed to it blinking out of existence.
From my small but nontrivial knowledge of Quine, he always struck me as having a critically wrong epistemology.
LW-style epistemology looks like this:
whereas Quine's seems more like
which seems to be missing most of the point.
His boat model always struck me as something confused that should be strongly modified or replaced by a Bayesian epistemology in which posterior follows logically and non-destructively from prior, but I may be in the minority in LW on this.
If you're wondering why I'm afraid of philosophy, look no further than the fact that this discussion is assigning salience to LW posts in a completely different way to I do.
I mean, it seems to me that where I think an LW post is important and interesting in proportion to how much it helps construct a Friendly AI, how much it gets people to participate in the human project, or the amount of confusion that it permanently and completely dissipates, all of this here is prioritizing LW posts to the extent that they happen to imply positions on famous ongoing philosophical arguments.
That's why I'm afraid to be put into any philosophical tradition, Quinean or otherwise - and why I think I'm justified in saying that their cognitive workflow is not like unto my cognitive workflow.
With this comment at least, you aren't addressing the list of 20+ useful contributions of mainstream philosophy I gave.
Almost none of the items I listed have to do with famous old "problems" like free will or reductionism.
Instead, they're stuff that (1) you're already making direct use of in building FAI, like reflective equilibrium, or (2) stuff that is almost identical to the 'coping with cognitive biases' stuff you've written about so much, like Bishop & Trout (2004), or (3) stuff that is dissolving traditional debates into the cognitive algorithms that produce them, which you seem to think is the defining hallmark of LW-style philosophy, or (4) generally useful stuff like the work on catastrophic risks coming out of FHI at Oxford.
I hope you aren't going to keep insisting that mainstream philosophy has nothing useful to offer after reading my list. On this point, it may be time for you to just say "oops" and move on.
After all, we already agree on most of the important points, like you said. We agree that philosophy is an incredibly diseased discipline. We agree that people shouldn't go out and read Quine. We agree that almost everyone should be reading s...
I can't believe how difficult it is to convince some people that some useful things come out of mainstream philosophy. To me, it's a trivial point.
If it's not immediately obvious how an argument connects to a specific implementable policy or empirical fact, default is to covertly interpret it as being about status.
Since there are both good and bad things about philosophy, we can choose to emphasize the good (which accords philosophers and those who read them higher status) or emphasize the bad (which accords people who do their own work and ignore mainstream philosophy higher status).
If there are no consequences to this choice, it's more pleasant to dwell upon the bad: after all, the worse mainstream philosophy does, the more useful and original this makes our community; the better mainstream philosophy does, the more it suggests our community is a relatively minor phenomenon within a broader movement of other people with more resources and prestige than ourselves (and the more those of us whose time is worth less than Eliezer's should be reading philosophy journals instead of doing something less mind-numbing).
I think this community is smart enough to avoid many such biases if...
Personally, I'm finding that avoiding anthropomorphising humans, i.e. ignoring the noises coming out of their mouths in favour of watching their actions, pays off quite well, particularly when applied to myself ;-) I call this the "lump of lard with buttons to push" theory of human motivation. Certainly if my mind had much effect on my behaviour, I'd expect to see more evidence than I do ...
"lump of lard with buttons to push"
I take exception to that: I have a skeletal structure, dammit!
No, they just look like they're doing it; saying humans are athropomorphizing would attribute more intentionality to humans than is justified by the data.
Okay, I read it. It's funny how Dennett's criticism of Skinner partially mirrors Luke's criticism of Eliezer. Because Skinner uses terminology that's not standard in philosophy, Dennett feels he needs to be "spruced up".
"Thus, spruced up, Skinner's position becomes the following: don't use intentional idioms in psychology" (p. 60). It turns out that this is Quine's position and Dennett sort of suggests that Skinner should just shut up and read Quine already.
Ultimately, I can understand and at least partially agree with Dennett that Skinner goes too far in denying the value of mental vocabulary. But, happily, this doesn't significantly alter my belief in the value of Skinner type therapy. People naturally tend to err in the other direction and ascribe a more complex mental life to my daughter than is useful in optimizing her therapy. And I still think Skinner is right that objections to behaviorist training of my daughter in the name of 'freedom' or 'dignity' are misplaced.
Anyway, this was a useful thing to read - thank you, ciphergoth!
Thanks so much. I didn't know about Quine, and from what you've quoted it seems quite clearly in the same vein as LessWrong.
Also, out of curiosity, do you know if anything's been written about whether an agent (natural or artificial) needs goals in order to learn? Obviously humans and animals have values, at least in the sense of reward and punishment or positive and negative outcomes -- does anyone think that this is of practical importance for building processes that can form accurate beliefs about the world?
What you care about determines what your explorations learn about. An AI that didn't care about anything you thought was important, even instrumentally (it had no use for energy, say) probably wouldn't learn anything you thought was important. A probability-updater without goals and without other forces choosing among possible explorations would just study dust specks.
I'm highly skeptical. I suspect that you may have failed to distinguish between sensory empiricism, which is a large standard movement, and the kind of thinking embodied in How An Algorithm Feels From the Inside which I've never seen anywhere else outside of Gary Drescher (and rumors that it's in Dennett books I haven't read).
Simple litmus test: What is the Quinean position on free will?
"It's nonsense!" = what I think standard "naturalistic" philosophy says
"If the brain uses the following specific AI-ish algorithms without conscious awareness of it, the corresponding mental ontology would appear from the inside to generate the following intuitions and apparent impossibilities about 'free will'..." = Less Wrong / Yudkowskian
Eliezer,
I'm not trying to say that you haven't made genuine contributions. Making genuine contributions in the Quinean path is what I mean when you say your work is part of that movement. And certainly, you speak a different language - the language of algorithms and AI rather than that of analytic philosophy. (Though there are quite a few who are doing philosophy in the language of AI, too: Judea Pearl is a shining example.)
'How an algorithm feels from the inside' is an important insight - an important way of seeing things. But your factual claims about free will are not radical. You agree with all naturalists that we do not have libertarian free will. We have no power to cause effects in the world without ourselves being fully caused, because we are fully part of nature. And you agree with naturalists that we are, nonetheless, able to deliberate about our actions. And that deliberation can, of course, affect the action we eventually choose. Our beliefs and desires affect our decisions, too.
Your differences with Quine look, to me at least, more like the differences that Quinean naturalists have with each other, rather than the differences that Quinean naturalists have with intuitionists and theists and postmodernists and phenomenologists, or even non-Quinean "naturalists" like Frank Jackson and David Chalmers.
Luke,
From my perspective, the idea that we do not have libertarian free will is too obvious to be interesting. If you want to claim that places me in a particular philosophical camp, fine, but that doesn't mean they do the same sort of cognitive labor I do when I'm doing philosophy. I knew there wasn't libertarian free will the instant I first considered the problem, at I think maybe age fourteen or thereabouts; if that made me a master philosopher, great, but to me it seems like the distance from there to being able to resolve the algorithms of the brain into their component parts was the interesting part of the journey.
(And Judea Pearl I have quite well acknowledged as an explicit shoulder to stand upon, but so far as I know he's another case of an AI researcher coming in from outside and solving a problem where philosophers just spun their wheels because they didn't think in algorithms.)
I did not put you in the Quinean camp merely because of your agreement about libertarian free will. I listed about a dozen close comparisons on matters that are highly controversial in mainstream philosophy. And I placed special emphasis on your eerily echo-ish defense of Quine's naturalized epistemology, which is central to both your philosophy and his.
I agree with you about Judea Pearl coming from AI to solve problems on which philosophers had been mostly stalled for centuries. Like Dennett says, AI researchers are doing philosophy - and really good philosophy - without really knowing it. Except for Pearl, actually. He does know he's doing philosophy, as becomes apparent in his book on causality, for example, where he is regularly citing the mainstream philosophical literature on the subject (alongside statistics and AI and so on).
Eliezer,
I don't get it. Your comment here doesn't respond to anything I said in my previous comment. The first sentence of my previous comment is: "I did not put you in the Quinean camp merely because of your agreement about libertarian free will."
I think Eliezer is suggesting that all the things you've mentioned that distinguish Quinean naturalists from other philosophers are similarly basic, and that "LW-style philosophy" takes (what turns out to be) Quinean naturalism as a starting point and then goes on to do things that no one working in mainstream philosophy has thought of.
In other words, that the problem with mainstream philosophy isn't that it's all wrong, but that much of it is wrong and that the part that isn't wrong is mostly not doing anything interesting with its not-wrongness.
(I make no comment on whether all, or some, or none, of that is correct. I'm just hoping to reduce the amount of talking-past-one-another here.)
Eliezer is suggesting that the Quineans are "not doing anything interesting with [their] not-wrongness" after being aware of the field for all of an hour and a half?!
Seems to me less like that and more like, "this Euclid fellow was brilliant", followed by a list of things that Euclid proved before anybody else proved. Timing matters here. It's no coincidence that before Quine came along, the clever Eliezers were not taking Quinean naturalism for granted.
For another analogy, if someone came along and told you, "this Hugh Everett fellow was brilliant! Here, read this paper in which he argues that the wave function never collapses", would you say, "well, Eliezer already went through that a few years ago; there's still no evidence that Everett made any worthwhile contribution"?
Eliezer's response does not. It looks like the response of one who feels their baby, LW style philosophy, is under attack. But it isn't.
Methinks Eliezer needs to spend more time practicing the virtues of scholarship by actually reading much of the philosophy that he is critiquing. His assessments of "naturalistic" philosophy seem like straw men. Furthermore, from a psychological perspective, it seems like Eliezer is trying to defend his previously made commitments to "LW-Style philosophy" at all costs. This is not the mark of true rationality - true rationality admits challenges to previous assumptions.
It doesn't matter how many verbal-type non-dissolved questions we agree on apart from that. I'm taking free will as an exemplar and saying, "But it's all like that, so far as I've been able to tell."
I'm not sure what you mean by this. Are you saying that my claim that LW-style philosophy shares many central assumptions with Quinean naturalism in contrast to most of philosophy doesn't hinge on whether or not I can present a long list of things on which LW-style philosophy and Quinean naturalism agree on, in contrast to most of philosophy?
I suspect that's not what you're saying, but then... what do you think it was that I was claiming in the first place?
Or, another way to put it: Which sentence of my original article are you disagreeing with? Do you disagree with my claim that "standard Less Wrong positions on philosophical matters have been standard positions in a movement within mainstream philosophy for half a century"? Or perhaps you disagree with my claim that "Less Wrong-style philosophy is part of a movement within mainstream philosophy to massively reform philosophy in light of recent cognitive science - a movement that has been active for at leas...
Are you saying that my claim that LW-style philosophy shares many central assumptions with Quinean naturalism in contrast to most of philosophy doesn't hinge on whether or not I can present a long list of things on which LW-style philosophy and Quinean naturalism agree on, in contrast to most of philosophy?
I'm saying that the claim that LW-style philosophy shares many assumptions with Quinean naturalism in contrast to most of philosophy is unimportant, thus, presenting the long list of basic assumptions on which LW-style and Quinean naturalism agree is from my perspective irrelevant.
Do you disagree with my claim that "standard Less Wrong positions on philosophical matters have been standard positions in a movement within mainstream philosophy for half a century"?
Yes. What I would consider "standard LW positions" is not "there is no libertarian free will" but rather "the philosophical debate on free will arises from the execution of the following cognitive algorithms X, Y, and Z". If the latter has been a standard position then I would be quite interested.
...Or perhaps you disagree with my claim that "Less Wrong-style philosophy
I'm saying that the claim that LW-style philosophy shares many assumptions with Quinean naturalism in contrast to most of philosophy is unimportant...
Well, it's important to my claim that LW-style philosophy fits into the category of Quinean naturalism, which I think is undeniable. You may think Quinean naturalism is obvious, but well... that's what makes you a Quinean naturalist. Part of the purpose of my post is to place LW-style philosophy in the context of mainstream philosophy, and my list of shared assumptions between LW-style philosophy and Quinean philosophy does just that. That goal by itself wasn't meant to be very important. But I think it's a categorization that cuts reality near enough the joints to be useful.
What I would consider "standard LW positions" is not "there is no libertarian free will" but rather "the philosophical debate on free will arises from the execution of the following cognitive algorithms X, Y, and Z". If the latter has been a standard position then I would be quite interested.
Then we are using the word "standard" in different ways. If I were to ask most people to list some "standard LW positions&q...
Assume naturalism! Move on! NEXT!
Yes, that's what most Quinean naturalists are doing...
Can I expect a reply to my claim that a central statement of your above comment was both clearly false and misrepresented Quinean naturalism? I hope so. I'm also still curious to hear your response to the specific example I've now given several times of how even non-naturalistic philosophy can provide useful insights that bear directly on your work on Friendly AI (the "extrapolation" bit).
As for expecting naturalistic philosophy to teach very bad habits of thought: That has some plausibility. But it is hard to argue about with any precision. What's the cost/benefit analysis on reading naturalistic philosophy after having undergone significant LW-rationality training? I don't know.
But I will point out that reading naturalistic philosophy (1) deconverted me from fundamentalist Christianity, (2) led me to reject most of standard analytic philosophy, (3) led me to almost all of the "standard" (in the sense I intended above) LW positions, and (4) got me reading and loving Epistemology and the Psychology of Human Judgment and Good and Real (two philosophy books that could just as...
Thanks for your reply.
On whether people can benefit from reading philosophy outside of Less Wrong and AI books, we simply disagree.
Your response on misrepresenting Quinean naturalism did not reply to this part: "Quinean naturalists don't just discuss the fact that cognitive biases affect philosophers. Quinean naturalists also discuss how to do philosophy amidst the influence of cognitive biases. That very question is a major subject of your writing on Less Wrong, so I doubt you see no value in it."
As for an example of dissolving certain questions into cognitive algorithms, I'm drafting up a post on that right now. (Actually, the current post was written as a dependency for the other post I'm writing.)
On CEV and extrapolation: You seem to agree that the distinction is useful, because you've used it yourself elsewhere (you just weren't going into so much detail in the CEV paper). But that seems to undermine your point that valuable insights are not to be found in mainstream philosophy. Or, maybe that's not your claim. Maybe your claim is that all the valuable insights of mainstream philosophy happen to have already shown up on Less Wrong and in AI textbooks. Either way, I once again simply disagree.
I doubt that you picked up all the useful philosophy you have put on Less Wrong exclusively from AI books.
It might be useful, if only for gaining status and attention and funding, to connect your work directly to one or several academic fields. To present it as a synthesis of philosophy, computer science, and cognitive science (or some other combination of your choice.) When people ask me what LessWrong is, I generally say something like "It's philosophy from a computer scientist's perspective." Most people can only put a mental label on something when they have a rough idea of what it's like, and it's not practical to say, "Well, our work isn't like anything."
That doesn't mean you have to hire philosophers or join a philosophy department; it might not mean that you, personally, have to do anything. But I do think that more people would be interested, and have a smaller inferential distance, if LW ideas were generally presented as related to other disciplines.
Just taking the example I happen to know about, Sarah-Jane Leslie works on the meaning of generics. (What do we mean when we say "Tigers have stripes" ? All tigers? Most tigers? Normal tigers? But then how do we account for true statements like "Tigers eat people" when most tigers don't eat people, or "Peacocks have colorful tails" when female peacocks don't have colorful tails?) She answers this question directly using evidence from cognitive science. I think it counts as question-dissolving.
When I read your first post here, my mind immediately went to You're Entitled to Arguments, But Not (That Particular) Proof. I gave you you the benefit of the doubt since you called it a 'litmus test' (however arbitrary), but you seem to have anchored on that. If your work is in substantial agreement with an established field in philosophy, that means there are more intelligent people who could become allies, and a store of knowledge from where valuable insights could come. I don't know why you are looking this particular gift horse in the mouth.
Physicalism and the rejection of free will are both majority positions in Anglophone philosophy, actually, but I agree that agreement on those points doesn't put someone on the shelf next to AIMA.
Physicalism and the rejection of free will are both majority positions in Anglophone philosophy
Regarding physicalism, I don't entirely trust that survey.
Firstly, most of those who call themselves physicalists nevertheless think that qualia exist and are Deeply Mysterious, such that one cannot deduce a priori, from objective physical facts, that Alfred isn't a zombie or that Alfred and Bob aren't qualia-inverted with respect to each other.
Secondly, in very recent years - 90s into the new century - I think there's been a rising tide of antimaterialism. Erstwhile physicalists such as Jaegwon Kim have defected. Anthologies are published with names like "The Waning of Materialism".
As the survey itself tell us, only 16% accept or lean towards "zombies are inconceivable".
This is all consistent with my experience in internet debates, where it seems that most upcoming or wannabe philosophers who have any confident opinions on the matter are antimaterialists.
All good points. I take back the claim that physicalism is a majority position; that is under serious doubt.
How sad! :(
Possibly you should state your hypothesis ahead of time and define what would count (or have counted in the past) as a worthwhile contribution to LW-style rationalism from within the analytic philosophy community.
Then we would have a concrete way to decide the question of whether analytic philosophy has contributed anything in the past, or contributes anything in the future.
It also might turn out in the process of formalising your definition of what counts as a worthwhile contribution that nothing outside of your specific field of AI research counts for you, which would in itself be a worthwhile realisation.
Acknowledging my own biases here, I'm an analytic philosopher who mostly teaches scientific methodology and ethics (with a minor side interest in statistics) and my reaction to perusing the LW content was that there were some very interesting and valuable nuggets here for me to fossick out but that the bulk of the content wasn't new or non-obvious to me.
Possibly there is so little for you in philosophy that has real novelty or value because there is already enormous overlap between what you do and what is done in the relevant subset of philosophy.
Being a philosopher makes you a...
the kind of thinking embodied in How An Algorithm Feels From the Inside which I've never seen anywhere else outside of Gary Drescher (and rumors that it's in Dennett books I haven't read).
Dennett is one of the leaders of mainstream philosophy. If it's in Dennett, Luke wins.
what I think standard "naturalistic" philosophy says
How did you acquire your beliefs about what standard "naturalistic" philosophy says? I have this impression that it was from outside caricatures rather than philosophers themselves.
Remember Scott Aaronson's critique of Stephen Wolfram? You seem at risk of being in a similar position with respect to mainstream analytic philosophy as Wolfram was with respect to mainstream science.
Discussions of priority are boring. If Quinean naturalism has insights relevant to LW, let's hear them!
What I'm saying is that Less Wrong shouldn't ignore mainstream philosophy.
What I demonstrated above is that, directly or indirectly, Less Wrong has already drawn heavily from mainstream philosophy. It would be odd to suggest that the progress in mainstream philosophy that Less Wrong has already made use of would suddenly stop, justifying a choice to ignore mainstream philosophy in the future.
As for naturalistic philosophy's insights relevant to LW, they are forthcoming. I'll be writing some more philosophical posts in the future.
And actually, my statistical prediction rules post came mostly from me reading a philosophy book (Epistemology and the Psychology of Human Judgment), not from reading psychology books.
I'll await your next post, but in retrospect you should have started with the big concrete example of mainstream philosophy doing an LW-style dissolution-to-algorithm not already covered on LW, and then told us that the moral was that we shouldn't ignore mainstream philosophy.
I did the whole sequence on QM to make the final point that people shouldn't trust physicists to get elementary Bayesian problems right. I didn't just walk in and tell them that physicists were untrustworthy.
If you want to make a point about medicine, you start by showing people a Bayesian problem that doctors get wrong; you don't start by telling them that doctors are untrustworthy.
If you want me to believe that philosophy isn't a terribly sick field, devoted to arguing instead of facing real-world tests and admiring problems instead of solving them and moving on, whose poison a novice should avoid in favor of eating healthy fields like settled physics (not string theory) or mainstream AI (not AGI), you're probably better off starting with the specific example first. "I disagree with your decision not to cover terminal vs. instrumental in CEV" doesn't cover it, and neither does "Quineans agree the world is made of atoms". Show me this field's power!
Eliezer,
When I wrote the post I didn't know that what you meant by "reductionist-grade naturalistic cognitive philosophy" was only the very narrow thing of dissolving philosophical problems to cognitive algorithms. After all, most of the useful philosophy you've done on Less Wrong is not specifically related to that very particular thing... which again supports my point that mainstream philosophy has more to offer than dissolution-to-algorithm. (Unless you think most of your philosophical writing on Less Wrong is useless.)
Also, I don't disagree with your decision not to cover means and ends in CEV.
Anyway. Here are some useful contributions of mainstream philosophy:
Quine's naturalized epistemology. Epistemology is a branch of cognitive science
Saying this may count as staking an exciting position in philosophy, already right there; but merely saying this doesn't shape my expectations about how people think, or tell me how to build an AI, or how to expect or do anything concrete that I couldn't do before, so from an LW perspective this isn't yet a move on the gameboard. At best it introduces a move on the gameboard.
Tarski on language and truth.
I know Tarski as a mathematician and have acknowledged my debt to him as a mathematician. Perhaps you can learn about him in philosophy, but that doesn't imply people should study philosophy if they will also run into Tarski by doing mathematics.
Chalmers' formalization of Good's intelligence explosion argument...
...was great for introducing mainstream academia to Good, but if you compare it to http://wiki.lesswrong.com/wiki/The_Hanson-Yudkowsky_AI-Foom_Debate then you'll see that most of the issues raised didn't fit into Chalmers's decomposition at all. Not suggesting that he should've done it differently in a first paper, but still, Chalmers's formalization doesn't yet represent most of th...
when IIRC, most of the philosophers were claiming at this point that causality was a mere illusion of correlation?
That this isn't at all the case should be obvious even if the only thing you've read on the subject is Pearl's book. The entire counterfactual approach is due to Lewis and Stalnaker. Salmon's theory isn't about correlation either. Also, see James Woodward who has done very similar work to Pearl but from a philosophy department. Pearl cites all of them if I recall.
As for the others: Yeah, we seem to agree that useful work does sometimes come from philosophy, but that it mostly doesn't, and people are better off reading statistics and AI and cognitive science, like I said. So I'm not sure there's anything left to argue.
I'd like to emphasize, to no one in particular, that the evaluation that seems to be going on here is about whether or not reading these philosophers is useful for building a Friendly recursively self-improving artificial intelligence. While thats a good criteria for whether or not Eliezer should read them, failure to meet this criteria doesn't render the work of the philosopher valueless (really! it doesn't!). The question "is philosophy helpful for researching AI" is not the same as the question "is philosophy helpful for a rational person trying to better understand the world".
Tarski did philosophical work on truth? Apart from his mathematical logic work on truth?
Okay, now you're just drawing lines around what you don't like and calling everything in that box philosophy.
Should we just hold a draft? With the first pick the philosophers select... Judea Pearl! What? whats that? The mathematicians have just grabbed Alfred Tarski from right under the noses the of the philosophers!
Yet to Wikipedia, Tarski is a mathematician. Period. Philosophy is not mentioned.
This sort of thing is less a fact about the world and more an artifact of the epistemological bias in English Wikipedia's wording and application of its verifiability rules. en:wp's way of thinking started at computer technology - as far as I can tell, the first field in which Wikipedia was the most useful encyclopedia - and went in concentric circles out from there (comp sci, maths, physics, the other sciences); work in the humanities less than a hundred or so years old gets screwed over regularly. This is because the verifiability rules have to more or less compress a degree's worth of training in sifting through human-generated evidence into a few quickly-comprehensible paragraphs, which are then overly misapplied by teenage science geek rulebots who have an "ugh" reaction to fuzzy subjects.
This is admittedly a bit of an overgeneralisation, but this sort of thing is actually a serious problem with Wikipedia's coverage of the humanities. (Which I'm currently researching with the assistance of upset academics in the area in order to make a suitable amount of targeted fuss about.)
tl;dr: that's stronger evidence of how Wikipedia works than of how the world works.
When I wrote the post I didn't know that what you meant by "reductionist-grade naturalistic cognitive philosophy" was only the very narrow thing of dissolving philosophical problems to cognitive algorithms.
No, it's more than that, but only things of that level are useful philosophy. Other things are not philosophy or more like background intros.
Amy just arrived and I've got to start book-writing, but I'll take one example from this list, the first one, so that I'm not picking and choosing; later if I've got a moment I'll do some others, in the order listed.
Funny you should mention that.
There is this incredibly toxic view of predicate logic that I first encountered in Good Old-Fashioned AI. And then this entirely different, highly useful and precise view of the uses and bounds of logic that I encountered when I started studying mathematical logic and learned about things like model theory.
Now considering that philosophers of the sort I inveighed against in "against modal logic" seem to talk and think like the GOFAI people and not like the model-theoretic people, I'm guessing that the GOFAI people made the terrible, horrible, no good, ver...
You may enjoy the following exchange between two philosophers and one mathematician.
Bertrand Russell, speaking of Godel's incompleteness theorem, wrote:
It made me glad that I was no longer working at mathematical logic. If a given set of axioms leads to a contradiction, it is clear that at least one of the axioms must be false.
Wittgenstein dismissed the theorem as trickery:
Mathematics cannot be incomplete; any more than a sense can be incomplete. Whatever I can understand, I must completely understand.
Godel replied:
Russell evidently misinterprets my result; however, he does so in a very interesting manner... In contradistinction Wittgenstein... advances a completely trivial and uninteresting misinterpretation.
According to Gleick (in The Information), the only person who understood Godel's theorem when Godel first presented it was another mathematician, Neumann Janos, who moved to the USA and began presenting it wherever he went, by then calling himself John von Neumann.
The soundtrack for Godel's incompleteness theorem should be, I think, the last couple minutes of 'Ludus' from Tabula Rasa by Arvo Part.
I've been wondering why von Neumann didn't do much work in the foundations of mathematics. (It seems like something he should have been very interested in.) Your comment made me do some searching. It turns out:
John von Neumann was a vain and brilliant man, well used to putting his stamp on a mathematical subject by sheer force of intellect. He had devoted considerable effort to the problem of the consistency of arithmetic, and in his presentation at the Konigsberg symposium, had even come forward as an advocate for Hilbert's program. Seeing at once the profound implications of Godel's achievement, he had taken it one step further—proving the unprovability of consistency, only to find that Godel had anticipated him. That was enough. Although full of admiration for Godel—he'd even lectured on his work—von Neumann vowed never to have anything more to do with logic. He is said to have boasted that after Godel, he simply never read another paper on logic. Logic had humiliated him, and von Neumann was not used to being humiliated. Even so, the vow proved impossible to keep, for von Neumann's need for powerful computational machinery eventually forced him to return to logic.
ETA: Am I ...
Of course, since this is a community blog, we can have it both ways. Those of us interested in philosophy can go out and read (and/or write) lots of it, and we'll chuck the good stuff this way. No need for anyone to miss out.
There is this incredibly toxic view of predicate logic that I first encountered in Good Old-Fashioned AI.
I'd be curious to know what that "toxic view" was. My GOFAI academic advisor back in grad school swore by predicate logic. The only argument against that I ever heard was that proving or disproving something is undecidable (in theory) and frequently intractible (in practice).
And then this entirely different, highly useful and precise view of the uses and bounds of logic that I encountered when I started studying mathematical logic and learned about things like model theory.
Model theory as opposed to proof theory? What is it you think is great about model theory?
...Now considering that philosophers of the sort I inveighed against in "against modal logic" seem to talk and think like the GOFAI people and not like the model-theoretic people, I'm guessing that the GOFAI people made the terrible, horrible, no good, very bad mistake of getting their views of logic from the descendants of Bertrand Russell who still called themselves "philosophers" instead of those descendants who considered themselves part of the thriving edifice of mathematics.
If that's what people got from it, then I'll add some clarifications to my original.
FWIW, what I got from your original post was not "LW readers should all go out and start reading mainstream philosophy," but rather "LW is part of a mainstream philosophical lineage, whether its members want to acknowledge that or not."
Meh. Historical context can help put things in perspective. You've done that plenty of times in your own posts on Less Wrong. Again, you seem to be holding my post to a different standard of usefulness than your own posts. But like I said, I don't recommend anybody actually read Quine.
It seems a shame to leave this list with several useful cites as a comment, where it is likely to be missed. Not sure what to suggest - maybe append it to the main article?
I did the whole sequence on QM to make the final point that people shouldn't trust physicists to get elementary Bayesian problems right.
Unfortunately for your argument in that sequence, very few actual physicists see the interpretation of quantum mechanics as a choice between "wavefunctions are real, and they collapse" and "wavefunctions are real, and they don't". I think life set you up for that choice because you got some of your early ideas about QM from Penrose, who does advocate a form of objective collapse theory. But the standard interpretation is that the wavefunction is not the objective state of the system, it is a tabulation of dispositional properties (that is philosophical terminology and would be unfamiliar to physicists, but it does express what the Copenhagen interpretation is about).
I might object to a lot of what physicists say about the meaning of quantum mechanics - probably the smartest ones are the informed realist agnostics like Gerard 't Hooft, who know that an observer-independent objectivity ought to be restored but who also know just how hard that will be to achieve. But the interpretation of quantum mechanics is not an "elementary Bayesian problem", nor is it an elementary problem of any sort. Given how deep the quantumness of the world goes, and the deep logical interconnectedness of things in physics, the correct explanation is probably one of the last fundamental facts about physics that we will figure out.
I doubt it. In fact I'm pretty certain that Quine had nothing to do with 'the origins of LW rationality'. I came to many (though by no means all) of the same conclusions as Eliezer independently, some of them in primary school, and never heard of Quine until my early 20s. What I had read - and what it's apparent Eliezer had read - was an enormous pile of hard science fiction, Feynman's memoirs, every pop-science book and issue of New Scientist I could get my hands on and, later, Feynman's Lectures In Physics. If you start out with a logical frame of mind, and fill that mind up with that kind of stuff, then the answers to certain questions come out as just "that's obvious!" or "that's a stupid question!" Enough of them did to me that I'm pretty certain that Eliezer also came to those conclusions (and the others he's come to and written about) independently.
Timing argues otherwise. We don't see Quine-style naturalists before Quine; we see plenty after Quine.
Eliezer doesn't recognize and acknowledge the influence? He probably wouldn't! People to a very large extent don't recognize their influences. To give just a trivial example, I have often said something to someone, only to find them weeks later repeating back to me the very same thing, as if they had thought of it. To give another example, pick some random words from your vocabulary - words like "chimpanzee", "enough", "unlikely". Which individual person taught you each of these words (probably by example), or which set of people? Do you remember? I don't. I really have no idea where I first picked up any bit of my language, with occasional exceptions.
For the most part we don't remember where exactly it was that we picked up this or that idea.
Of course, if Eliezer says he never read Quine, I don't doubt that he never read Quine. But that doesn't mean that he wasn't influenced by Quine. Quine influenced a lot of people, who influenced a lot of other people, who influenced still more people, some of whom could very easily have influenced Eliezer without Eliezer having the slightest notion that the influence originated with Quine.
It's hard to trace influence. What's not so hard is to observe timing. Quine comes first - by decades.
This is indeed why most philosophy is useless. But I've asserted that most philosophy is useless for a long time. This wouldn't explain why philosophy would nevertheless make useful progress up until the 60s or 80s or 2000s and then suddenly stop. That suggestion remains to be explained.
I don't get it. When low-hanging fruit is covered on Less Wrong, it's considered useful stuff. When low-hanging fruit comes from mainstream philosophy, it supposedly doesn't help show that mainstream philosophy is useful. If that's what's going on, it's a double standard, and a desperate attempt to "show" that mainstream philosophy isn't useful.
Also, saying "Well, we already know about lots of mainstream philosophy that's useful" is direct support for the central claim of my original post: That mainstream philosophy can be useful and shouldn't be ignored.
In undergrad I had to read Quine's From Stimulus to Science for one of my philosophy classes, and I remember thinking "so what's your point?" It seemed like what Quine really needed to do in that work was talk about induction, but he just skirted the issue. Have you read it? What's your take? This was my only real exposure to Quine, so it's probably part of the reason I dismissed him.
(It's been a couple years since I've read it, so my memory may be off or I might have a different view if I read it now.)
I think Quine's original works are hard to read, and not the best presentation of his own work. I recommend instead Quine: a guide for the perplexed.
In general, I think primary literature is over-recommended for initial learning. There is almost always better coverage of the subject in secondary literature.
So I stumbled on these instructions:
Go to a random wikipedia article. Click on the first link (skip parentheses). Repeat. You will always end up on 'Philosophy.'
Below is a list of the random articles I began from, and how long it took me to get to the Philosophy article.
Gymnasium Philippinum: 11
Brnakot: 23
Ohrenbach: 11
Vrijburg: 24
The Love Transcendent: 14
2010 in tennis: 13
Cross of All Nations: 24
List of teams and cyclists in the 2003 Tour de France: 14
Anton Ehmann: 19
Traveling carnival: 25
Frog: 13
Some, however, go into an immediate loop, for exampl...
This seems to be saying that Quinean philosophy reached (correct) conclusions similar to Less Wrong, and that since it came first it probably influenced LW, directly or indirectly, and therefore, we should study Quinean philosophy. But this does not follow; if LW and Quine say the same things, and either LW is better written or we've already read it, then this is a reason not to read Quine, because of the duplication. The implied argument seems to be: Quine said these things first => Quine deserves prestige => We should read Quine. But prestige alone is not a sufficient reason to read anything.
No, I advise against reading Quine. I only said above that rationalists should not ignore mainstream (Quinean) philosophy. That's a much weaker claim than the one you've attributed to me. Much of LW is better-written and more informed of the latest science than some of the best Quinean philosophy being written today.
What I'm claiming is that Quinean philosophy has made, and continues to make, useful contributions, and thus shouldn't be ignored. I have some examples of useful contributions from Quinean philosophy here.
I don't advocate reading Quine directly, but rather Quinean philosophy. For example Epistemology and the Psychology of Human Judgment, which reads like a series of Less Wrong blog posts, but covers lots of material not yet covered on Less Wrong. (I made a dent in this by transposing its coverage of statistical prediction rules into a Less Wrong post.)
And I don't advocate it for everyone. Doing research in philosophy is my specialty, but I don't think Eliezer should waste his time poring through philosophy journals for useful insights. Nor should most people. But then, most people won't benefit from reading through books on algorithmic learning theory, either. That's why we have divisions of labor and expertise. The thing I'm arguing against is Eliezer's suggestion that people shouldn't read philosophy at all outside of Less Wrong and AI books.
Some questions and thoughts about this:
How is it that 'naturalism' is the L.W. philosophy? I am not a naturalist, as I understand that term. What is the prospect of fair treatment for a dissenter to the L.W. orthodoxy?
Where does Quine talk about postmodernism, or debates about the meanings of terms like 'knowledge'? If a reference is available it'd be appreciated.
What exactly do you understand by 'naturalism' - what does it commit you to? Pointing to Quine et. al. gives some indication, but it should not be assumed that there is no value, if being a
One philosopher whose work it would be extremely interesting to see analyzed from a LW-style perspective is Max Stirner. Stirner has, in my opinion, been unfairly neglected in academic philosophy, and to the extent that his philosophy has been given attention, it was mostly in various nonsensical postmodernist and wannabe-avantgardist contexts. However, a straightforward reading of his original work is a very rewarding intellectual exercise, and I'd really like to see a serious no-nonsense discussion of his philosophy.
I just read most of the comments in this thread and despite a general agreement that LW philosophy and Quinean philosophy have a whole lot in common, no one even suggested reading up on his critics. The lens that sees it's own flaws is fine and good(although always a more difficult task than one might expect), but what about the lens that exposes itself to the eyes of others and asks them what flaws they see?
And, especially with the replication crisis, shouldn't we all be gaining some awareness of the philosophical assumptions that inform our experim...
I find reading this post and the ensuing discussion quite interesting because I studied academic philosophy (both analytic and continental) for about 12 years at university. Then I changed course and moved into programming and math, and developed a strong interest thinking about AI safety.
I find this debate a bit strange. Academic philosophy has its problems, but it's also a massive treasure trove of interesting ideas and rigorous arguments. I can understand the feeling of not wanting to get bogged down in the endless minutia of academic philosophizing in ...
I just wanted to thank you for your continuous work, and especially for explicitly sparing us the work of sifting through all that philosophical tradition.
The stimulation of his sensory receptors is all the evidence anybody has had to go on, ultimately, in arriving at his picture of the world.
Isn't this missing our evolutionary prior, our instinct? And what argument can be made against privileged philosophical insight? That the map is not the territory is only partly true as the map is part of the territory and everyone can explore and alter their map and therefore alter the territory. That is what is happening when philosophers and mathematicians explore the abstract landscapes of mathematics and languag...
I suspect some philosophers of mind would reply 'Philosophy of mind makes AI studies honest'. Also, if you are averse to recommending the reading of Quine, at least recommend some of his critics. If your views are Quinean, surely you should at least have a look at the vast anti-Quinean literature out there?
After a proposed analysis or definition is overturned by an intuitive counterexample, the idea is to revise or replace the analysis with one that is not subject to the counterexample. Counterexamples to the new analysis are sought, the analysis revised if any counterexamples are found, and so on...
Interestingly, that sounds a lot like (an important part of) how linguistics research works. Of course, it's a problem for philosophy because it doesn't see itself as a cognitive science like linguistics does, and it endeavours to do other things with this approach than deducing the rules of the system that generates the intuitions.
Someone asked me via email:
How do you see the analytic/synthetic distinction relating to map/territory? I suspect I read the logical positivists with too much charity, because I fit their arguments into my conception of map and territory. Quine attacked the positivists' view with what I know you've said is a view much like what LessWrong holds.
I figured my answer will be helpful for others, too, so I'll post it here:
...The analytic/synthetic distinction is quite different than map/territory. The map/territory distinction is a metaphor that illustrat
Now that I've actually read some Quinean naturalism (and similar stuff), I'm gonna' downvote. Better than the surrounding papers, okay. Probably important steps forward. But not all that good, in the light of decades worth of higher expectations. The only associated things I would exempt (that I've read) are some of Pierce's writings.
Socrates definitely drew on some questionable intuitions in Plato's dialogues, but I think justice is a particularly slippery concept, in that it requires a prior conception of both the law and the good.
Legality is, for any sufficiently well-written laws, a purely factual matter. Does x break the law? Yes/no. Morality is, for any particular sufficiently well-written moral system, also a factual matter, but with more degrees of freedom. Is x good? Is x optimally good? Is x bad, but the best available option? Is the moral law, as written, itself good or opt...
Part of the sequence: Rationality and Philosophy
Despite Yudkowsky's distaste for mainstream philosophy, Less Wrong is largely a philosophy blog. Major topics include epistemology, philosophy of language, free will, metaphysics, metaethics, normative ethics, machine ethics, axiology, philosophy of mind, and more.
Moreover, standard Less Wrong positions on philosophical matters have been standard positions in a movement within mainstream philosophy for half a century. That movement is sometimes called "Quinean naturalism" after Harvard's W.V. Quine, who articulated the Less Wrong approach to philosophy in the 1960s. Quine was one of the most influential philosophers of the last 200 years, so I'm not talking about an obscure movement in philosophy.
Let us survey the connections. Quine thought that philosophy was continuous with science - and where it wasn't, it was bad philosophy. He embraced empiricism and reductionism. He rejected the notion of libertarian free will. He regarded postmodernism as sophistry. Like Wittgenstein and Yudkowsky, Quine didn't try to straightforwardly solve traditional Big Questions as much as he either dissolved those questions or reframed them such that they could be solved. He dismissed endless semantic arguments about the meaning of vague terms like knowledge. He rejected a priori knowledge. He rejected the notion of privileged philosophical insight: knowledge comes from ordinary knowledge, as best refined by science. Eliezer once said that philosophy should be about cognitive science, and Quine would agree. Quine famously wrote:
But isn't this using science to justify science? Isn't that circular? Not quite, say Quine and Yudkowsky. It is merely "reflecting on your mind's degree of trustworthiness, using your current mind as opposed to something else." Luckily, the brain is the lens that sees its flaws. And thus, says Quine:
Yudkowsky once wrote, "If there's any centralized repository of reductionist-grade naturalistic cognitive philosophy, I've never heard mention of it."
When I read that I thought: What? That's Quinean naturalism! That's Kornblith and Stich and Bickle and the Churchlands and Thagard and Metzinger and Northoff! There are hundreds of philosophers who do that!
Non-Quinean philosophy
But I should also mention that LW philosophy / Quinean naturalism is not the largest strain of mainstream philosophy. Most philosophy is still done in relative ignorance (or ignoring) of cognitive science. Consider the preface to Rethinking Intuition:
Conclusion
So Less Wrong-style philosophy is part of a movement within mainstream philosophy to massively reform philosophy in light of recent cognitive science - a movement that has been active for at least two decades. Moreover, Less Wrong-style philosophy has its roots in Quinean naturalism from fifty years ago.
And I haven't even covered all the work in formal epistemology toward (1) mathematically formalizing concepts related to induction, belief, choice, and action, and (2) arguing about the foundations of probability, statistics, game theory, decision theory, and algorithmic learning theory.
So: Rationalists need not dismiss or avoid philosophy.
Update: To be clear, though, I don't recommend reading Quine. Most people should not spend their time reading even Quinean philosophy; learning statistics and AI and cognitive science will be far more useful. All I'm saying is that mainstream philosophy, especially Quinean philosophy, does make some useful contributions. I've listed more than 20 of mainstream philosophy's useful contributions here, including several instances of classic LW dissolution-to-algorithm.
But maybe it's a testament to the epistemic utility of Less Wrong-ian rationality training and thinking like an AI researcher that Less Wrong got so many things right without much interaction with Quinean naturalism. As Daniel Dennett (2006) said, "AI makes philosophy honest."
Next post: Philosophy: A Diseased Discipline
References
Dennett (2006). Computers as Prostheses for the Imagination. Talk presented at the International Computers and Philosophy Conference, Laval, France, May 3, 2006.
Kahneman, Slovic, & Tversky (1982). Judgment Under Uncertainty: Heuristics and Biases. Cambridge University Press.
Nisbett and Ross (1980). Human Inference: Strategies and Shortcomings of Social Judgment. Prentice-Hall.
Rips (1975). Inductive judgments about natural categories. Journal of Verbal Learning and Behavior, 12: 1-20.
Rosch (1978). Principles of categorization. In Rosch & Lloyd (eds.), Cognition and Categorization (pp. 27-48). Lawrence Erlbaum Associates.
Rosch & Mervis (1975). Family resemblances: studies in the internal structure of categories. Cognitive Psychology, 8: 382-439.
Smith & Medin (1981). Concepts and Categories. MIT Press.