Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Eliezer_Yudkowsky comments on Less Wrong Rationality and Mainstream Philosophy - Less Wrong

106 Post author: lukeprog 20 March 2011 08:28PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (328)

You are viewing a single comment's thread.

Comment author: Eliezer_Yudkowsky 25 March 2011 08:33:15AM 17 points [-]

Note the way I speak with John Baez in the following interview, done months before the present post:


In terms of what I would advocate programming a very powerful AI to actually do, the keywords are “mature folk morality” and “reflective equilibrium”...

In terms of Google keywords, my brand of metaethics is closest to analytic descriptivism or moral functionalism...

I was happy to try and phrase this interview as if it actually had something to do with philosophy.

Although I actually invented the relevant positions myself, on the fly when FAI theory needed it, then Googled around to find the philosophical nearest neighbor.

The fact that you are skeptical about this, and suspect I suppose that I accidentally picked up some analytic descriptivism or mature folk morality elsewhere and then forgot I'd read about it, even though I hadn't gone anywhere remotely near that field of philosophy until I wanted to try speaking their language, well, that strikes at the heart of why all this praise of "mainstream" philosophy strikes me the wrong way. Because the versions of "mature folk morality" and "reflective equilibrium" and "analytic descriptivism" and "moral functionalism" are never quite exactly right, they are built on entirely different premises of argument and never quite optimized for Friendly-AI thinking. And it seems to me, at least, that it is perfectly reasonable to simply ignore the field of philosophy and invent all these things the correct way, on the fly, and look up the nearest neighbor afterward; some wheels are simple enough that they're cheaper to reinvent than to look up and then modify.

Can philosophers be useful? Yes. Is it possible and sometimes desirable to communicate with people who've previously read philosophy in philosophical standard language? Yes. Is Less Wrong a branch from the mighty tree of mainstream philosophy? No.

Comment author: lukeprog 25 March 2011 05:05:51PM 19 points [-]

With this comment, I think our disagreement is resolved, at least to my satisfaction.

We agree that philosophy can be useful, and that sometimes it's desirable to speak the common language. I agree that sometimes it is easier to reinvent the wheel, but sometimes it's not.

As for whether Less Wrong is a branch of mainstream philosophy, I'm not much interested to argue about that. There are many basic assumptions shared by Quinean philosophy and Yudkowskian philosophy in opposition to most philosophers, even down to some very specific ideas like naturalized epistemology that to my knowledge had not been articulated very well until Quine. And both Yudkowskian philosophy and Quinean naturalism spend an awful lot of time dissolving philosophical debates into cognitive algorithms and challenging intuitionist thinking - so far, those have been the main foci of experimental philosophy, which is very Quinean, and was mostly founded by one of Quine's students, Stephen Stich. Those are the reasons I presented Yudkowskian philosophy as part of the broadly Quinean movement in philosophy.

On the other hand, I'm happy to take your word for it that you came up with most of this stuff on your own, and only later figured out what the philosophers have been calling it, so in another way Yudkowskian philosophy is thoroughly divorced from mainstream philosophy - maybe even more than, say, Nassim Taleb's philosophical work.

And once we've said all that, I don't think any question remains about whether Less Wrong is really part of a larger movement in philosophy.

Anyway, thanks for this further clarification. I've learned a lot from our discussion. And I'm enjoying your interview with Baez. Cheers.

Comment author: ToddStark 17 December 2012 04:40:41AM 1 point [-]

On the general issue of the origin of various philosophical ideas, I had a thought. Perhaps we take a lot of our tacit knowledge for granted in our thinking about attributions. I suspect that abstract ideas become part of wider culture and then serve as part of the reasoning of other people without them explicitly realizing the role of those abstracts. For example, Karl Popper had a concept of "World 3" which was essentially the world of artifacts that are inherited from generation to generation and become a kind of background for the thinking of each successive generation who inherits that culure. That concept of "unconscious ideas" was also found in a number of other places (and has been of course for as far back as we can remember) and has been incorporated into many theories and explanations of varying usefulness. Some of Freud's ideas have a similar rough feel to them and his albeit unscientific ideas became highly influential in popular culture and influenced all sorts of things, including some productive psychology programs that emphasize influences outside of explcit awareness. Our thinking is given shape in part by a background that we aren't explicitly aware of and as a result we can'[t always make accurate attributions of intellectual history except in terms of what has been written down. Some of the influence happens outside of our awareness via various mechanisms of implicit or tacit learning. We know a lot more than we realize we know, we "stand on the shoulders of others" in a somewhat obscure sense as well as the more obvious one.

An important implication of this might be that our reasoning starts from assumptions and conceptual schemes that we don't really think about because it is "intuitive" and appears to each of us as "commonsense." However it may be that "commonsense" and "intuition" are forms of ubiquitous expertise that differ somewhat between people. If that is the case, then people reason from different starting points and perhaps can reason to different conclusions even when rigorously logical, and this would seemingly support a perspectivist view where logic is not by itself adequate to reconcile differences in opinion.

If that is the case, then it helps explain why we can't seem to get rid of some fundamental problems just by clarifying concepts and reasoning from evidence. Those operations are themselves shaped by a background. One of the important roles of philosophy may be to give a voice to some of that background, a voice which may not always be scientific (that is, empirical, testable, effectively communicated through mathematics). So it may not be the philosophers who actually make the ideas available ot us, but the philosophers who make them explicit outside of science.

I'm not saying that contradicts the possibly unique value of naturalistic and reductionistic approaches, systematization, etc., just that if we think of philosophy purely in utilitarian terms as a provider of new theories that feed science, we may miss the point of its role in culture and our tracking and understanding of the genesis of ideas.

Comment author: BobTheBob 27 March 2011 04:40:34PM 4 points [-]

You say,

the versions of "mature folk morality" and "reflective equilibrium" and "analytic descriptivism" and "moral functionalism" are never quite exactly right, they are built on entirely different premises of argument and never quite optimized for Friendly-AI thinking.

and that you prefer to "invent all these things the correct way".

From this and your preceding text I understand,

  • that philosophers have identified some meta-ethical theses and concepts similar to concepts and theses you've invented all by yourself,

  • that the philosophers' theses and concepts are in some way systematically defective or inadequate, and

  • that the arguments used to defend the theses are different than the arguments which you would use to defend them.

(I'm not sure what you mean in saying the concepts and theses aren't optimized for Friendly-AI thinking.)

You imply that you've done a comprehensive survey, to arrive at these conclusions. It'd be great if you could share the details. Which discussions of these ideas have you studied, how do your concepts differ from the philosophers', and what specifically are the flaws in the philosophers' versions? I'm not familiar with these meta-ethical theses but I see that Frank Jackson and Philip Pettit are credited with sparking the debate in philosophy - what in their thinking do you find inadequate? And what makes your method of invention (to use your term) of these things the correct one?

I apologize if the answers to these questions are all contained in your sequences. I've looked at some of them but the ones I've encountered do not answer these questions.

You disparage the value of philosophy, but it seems to me you could benefit from it. In another of your posts, 'How An Algorithm Feels From Inside', I came across the following:

When you look at a green cup, you don't think of yourself as seeing a picture reconstructed in your visual cortex - although that is what you are seeing - you just see a green cup. You think, "Why, look, this cup is green," not, "The picture in my visual cortex of this cup is green."

This is false - the claim, I mean, that when you look at a green cup, you are seeing a picture in your visual cortex. On the contrary, the thing you see is reflecting light, is on the table in front of you (say), has a mass of many grams, is made of ceramic (say), and on an on. It's a cup -it emphatically is not in your brainpan. Now, if you want to counter that I'm just quibbling over the meaning of the verb 'to see', that's fine - my point is that it is you who are using it in a non-standard way, and it behoves you to give a coherent explication of your meaning. The history of philosophical discussions suggests this is not an easy task. The root of the problem is the effort to push the subject/object distinction -which verbs of perception seem to require- within the confines of the cranium. Typically, the distinction is only made more problematic - the object of perception (now a 'picture in the visual cortex') still doesn't have the properties it's supposed to (greenness), and the subject doing the seeing seems even more problematic. The self is made identical to or resident within some sub-region of the brain, about which various awkward questions now arise. Daniel Dennett has criticized this idea as the 'Cartesian Theatre' model of perception.

Having talked to critics of philosophy before, I know such arguments are often met with considerable impatience and derision. They are irrelevant to the understanding being sought, a waste of time, etc. This is fine - it may be true, for many, including you. If this is so, though, it seems to me the rational course is simply to acknowledge it's concerns are orthogonal to your own, and if you seem to come into collision (as above), to show that your misleading metaphor isn't really doing any work, and hence is benign. In this case you aren't re-inventing the wheel in coming up with your own theories, but something altogether different - a skid, maybe.