Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Eliezer's Sequences and Mainstream Academia

90 Post author: lukeprog 15 September 2012 12:32AM

Due in part to Eliezer's writing style (e.g. not many citations), and in part to Eliezer's scholarship preferences (e.g. his preference to figure out much of philosophy on his own), Eliezer's Sequences don't accurately reflect the close agreement between the content of The Sequences and work previously done in mainstream academia.

I predict several effects from this:

  1. Some readers will mistakenly think that common Less Wrong views are more parochial than they really are.
  2. Some readers will mistakenly think Eliezer's Sequences are more original than they really are.
  3. If readers want to know more about the topic of a given article, it will be more difficult for them to find the related works in academia than if those works had been cited in Eliezer's article.

I'd like to counteract these effects by connecting the Sequences to the professional literature. (Note: I sort of doubt it would have been a good idea for Eliezer to spend his time tracking down more references and so on, but I realized a few weeks ago that it wouldn't take me much effort to list some of those references.)

I don't mean to minimize the awesomeness of the Sequences. There is much original content in them (edit: probably most of their content is original), they are engagingly written, and they often have a more transformative effect on readers than the corresponding academic literature.

I'll break my list of references into sections based on how likely I think it is that a reader will have missed the agreement between Eliezer's articles and mainstream academic work.

(This is only a preliminary list of connections.)

 

Obviously connected to mainstream academic work


Less obviously connected to mainstream academic work


I don't think Eliezer had encountered this mainstream work when he wrote his articles

Comments (153)

Comment author: Wei_Dai 15 September 2012 04:57:22AM 24 points [-]

Wow this is awesome. Some comments and questions:

  • Spohn's decision theory does look very similar to Eliezer's, but Spohn couldn't give a good argument for the plausibility of rational cooperation in one-shot PD (he tried in the 2003 paper) because his didn't have the concepts of decision making as an algorithm, and of logical correlation between instances of such algorithms.
  • The kind of AI cooperation discussed by Eliezer is not the type discussed as "program equilibrium". Instead "program equilibrium" is very similar (essentially the same?) as cousin_it's initial approach to AI cooperation, which he came up with in part due to dissatisfaction with Eliezer's approach. (cousin_it later moved on to "Lobian cooperation", which is closer to Eliezer's idea, and as far as anyone knows those results weren't previously discovered in academia.)
  • In your research, did you fail to find previous academic work for some elements of the sequences? In other words, which other elements are not (known to be) expositions or reinventions of previous academic work?
Comment author: lukeprog 15 September 2012 09:13:46PM 4 points [-]

The kind of AI cooperation discussed by Eliezer is not the type discussed as "program equilibrium".

Fixed, thanks.

In your research, did you fail to find previous academic work for some elements of the sequences?

I didn't look very hard. I merely thought about the stuff I already knew about, and then picked a subset of those things to list here.

Comment author: [deleted] 15 September 2012 01:00:38AM 19 points [-]

Some readers will mistakenly think that common Less Wrong views are more parochial than they really are.

This one is probably important. With non-LW newcomers to my meetup, I find explaining that we draw most of our ideas from LW kind of weird in the sense that 50% of my simulations of them conclude "these guys are some wacko internet cult". Only some of them come back.

I'd like to see at least some work on how to talk about LW without implying insularity. We can't just drop the LW affiliation, because nowhere else really compares (even if everything on LW exists somewhere else, it doesn't exist anywhere else all in the same place).

Comment author: RomeoStevens 15 September 2012 05:12:10AM *  31 points [-]

HI WE'RE AN INTERNET MEETUP GROUP!

The internet is low status due to the low barriers to entry. Mention higher status things than the internet.
Future of Humanity Institute at Oxford (Hey I've heard of Oxford)
Vinge (a published author many have heard of)
Center for Applied Rationality which does real things in real life with real people

Talk about cognitive science. Talk about economics. Talk about anything but the internet. LessWrong? Oh it's just for coordinating all the interesting people who are interested in these interesting things.

Comment author: lukeprog 16 September 2012 02:52:02PM *  16 points [-]

I'd like to see at least some work on how to talk about LW without implying insularity.

Name-drop like a motha...

  • "Did you read Predictably Irrational by Dan Ariely or Thinking Fast and Slow by Nobel laureate Daniel Kahneman? We study their field of predictable human thinking errors and try to figure out how best to apply those lessons to everyday human life so that we can learn how to make decisions that are more likely to achieve our goals.

  • "We talk some about Alan Turing's idea that machines could one day become smarter than humans, and how shortly thereafter we might expect them to become more powerful than humans. One of the mathematicians who worked with him to crack the German Enigma Code, I.J. Good, explained that a smarter-than-human machine could use its intelligence to improve its own inteligence. And since neuroscientists like Paul Glimcher at NYU and Kent Berridge at U Michigan are learning that what humans care about is incredibly complex, it's unlikely that we'll be able to figure out how to program smarter-than-human machines to respect every little detail of what we care about."

Comment author: siodine 16 September 2012 04:02:58PM 1 point [-]

Or, more meta-ly, you're not going to be very persuasive if you ignore pathos and ethos. I think this might be a common failure mode of aspiring rationalists because we feel we shouldn't have to worry about such things, but then we're living in the should-world rather than the real-world.

Comment author: Manfred 15 September 2012 01:20:46AM *  6 points [-]

Likewise, much of the Quantum Physics sequence can be found in quantum physics textbooks, e.g. Sakurai & Napolitano (2010).

I don't think Sakurai is the best reference here - most of an introductory QM book will be about what particles do in the presence of forces, and treats identical particles in a more complicated language because they can be either fermions or bosons.

A better text would be an introduction for people who want to do quantum computing - those people get to use all the nice abstractions and let the physicists worry about the particles in the presence of forces behind those abstractions :P An example I was able to dig up from a course syllabus was (Robert, not David) Griffiths' Consistent Quantum Theory.

EDIT: Ah, of course the best reference is Feynman's QED.

Comment author: Vaniver 15 September 2012 01:19:59AM 29 points [-]

Why is this only in discussion?

Comment author: [deleted] 15 September 2012 08:45:41PM 34 points [-]

Because the distinction between main and discussion is really confusing.

Comment author: Risto_Saarelma 16 September 2012 03:03:20PM 5 points [-]

This is good. Getting people to read lots of quality stuff by a wide variety of authors can put them on the path of being able to produce high quality output themselves, after overlearning the concepts and ways to present them from many viewpoints.

People who just stick with the sequences can end up parroting the surface jargon and alienating people who expect familiarity with a bit wider range of literature for someone whose opinion they would value.

Comment author: ChrisHallquist 18 September 2012 12:45:11AM 11 points [-]

Just want to give one piece of positive feedback: I've been meaning to get some recommendations for reading on many of these topics, and these citations are way awesomer than what I would've hoped to get before. Thanks, Luke!

Comment author: lukeprog 18 September 2012 07:17:27PM 7 points [-]

Thanks for bothering to give "lowly" positive feedback! :)

Comment author: brilee 15 September 2012 01:02:29PM 4 points [-]

It's posts like this that remind me that the sequences are vast, excellent, and most importantly of all, not particularly organized at the moment.

Every so often, Lukeprog or others will make a small effort towards collating the sequences, but the resulting product disappears into the ether of Discussion archives.

Talk is cheap, but somebody really needs to do something about the sequences to make them more accessible and visible to a newcomer. The LW wiki index of the sequence is incomplete, and seems like it hasn't been changed since 'Tetronian' created it six months ago.

Comment author: Rain 15 September 2012 04:27:44PM *  2 points [-]

They're compiling a book-format edition of the Sequences, and there's quite a bit of work into an alternate pop-sci edition.

Comment author: Filipe 15 September 2012 10:45:04AM *  8 points [-]

What about Drescher's Good and Real: Demystifying Paradoxes from Physics to Ethics? Eliezer said it's "pratically Less Wrong in book form."

Comment author: Manfred 15 September 2012 01:04:53PM 1 point [-]

Not a source, but definitely a parallel.

We have talked about Newcomb's problem with transparent boxes on here a few times - I'm pretty sure that's originally from Good and Real.

Comment author: betterthanwell 20 September 2012 12:09:33AM 0 points [-]

Yep. Gloriously lucid and quite readable book.
Encapsulates good chunks of the sequences.

Much more accessible than I had anticipated.

Comment author: Thrasymachus 18 September 2012 08:13:54AM 7 points [-]

Some readers will mistakenly think that common Less Wrong views are more parochial than they really are.

I think the parochialism comes from high handed smack-talk like "The obvious answer to philosophically recondite issue is X, and all you need to see this is obvious is our superior rationality". Best example here.

One of the easiest hard questions, as millennia-old philosophical dilemmas go. Though this impossible question is fully and completely dissolved on Less Wrong, aspiring reductionists should try to solve it on their own.

I get a similar vibe regarding QM (obviously many worlds), religion (obviously atheism), phil of mind (obviously reductionsim), and (most worrying) ethics and meta-ethics.

The fact the candidate views espoused are part of the academic mainstream doesn't defray the charge of parochialism due to the tup-thumping, uncharitable-to-opponents and generally under-argued way these views are asserted. Worse, it signals lack of competence on the part of LW: given the views of virtually all domain experts on any of these things, your degree of confidence is better explained by inferior, not superior knowledge, and even if you happen to get the right answer, I doubt you're p-reliable or tracking.

Comment author: ArisKatsaris 18 September 2012 08:31:34AM *  14 points [-]

I don't think there's much value in pretending that issues like God (and the absence thereof) or the compatibility between determinism and (any logically coherent view of) free will haven't been decisively answered.

Seriously now, the compatibility between free will and determinism is something that I was figuring out by myself back in junior high. Eliezer with his "Thou Art Physics" expressed it better and more compactly than I ever did to myself (I was instead using imagery of the style "we're the stories that write themselves", and this was largely inspired by Tolkien's Ainulindale, where the various gods sing a creation song that predicts all their future behaviour), but the gist is really obvious once you get rid of the assumption that determinism and free will must somehow be opposed.

In every discussion I've had since, in any forum, nobody who thinks them to be incompatible can describe even vaguely what "free will" would be supposed to look like if it does not contain determinism inside it.

Comment author: Thrasymachus 21 September 2012 02:58:07PM 6 points [-]

I think this is a case of exactly the problem I diagnosed above.

Compatibilism (and related views) have been mentioned at least since Hume, and have been discussed extensively in modern analytic philosophy. Although it commands a slender majority of philosophers of action, it is not like the entire philosophical community considers compatibilism obviously or decisively the 'right answer' (see here, and here for a long index of reasons/objections etc.). You'd be pretty hard pressed to find a single philosopher of action who considers free will a 'solved problem'.

Yet it seems the less wrong community considers it solved based on a sequence of blog posts which merely explicates compatibilism: I couldn't find any discussion of compatibilism which goes beyond undergrad philosophy level, no discussion of common objections to compatibilism, engagement with any thinkers arguing against, nothing.

The two best explanations I have for this is either compatibilism is just obvious and people of sufficient rationality can be confident that domain experts on free will who don't buy compatibilism are wrong, or that the LW 'solution' is frankly philosophically primitive but LWers are generally too far on the wrong side of the Dunning-Kruger effect to appreciate why it isn't the decisive answer to a 'millenia old philosophical dilemma' they think it is.

Surely the outside view would find the latter account much more plausible?

Comment author: ArisKatsaris 21 September 2012 03:43:46PM *  0 points [-]

Although it commands a slender majority of philosophers of action, it is not like the entire philosophical community considers compatibilism obviously or decisively the 'right answer'

Thanks for that poll. It's a slender majority, but a very strong plurality, since the next most favourite option is less than half as popular, and if you examine only the 'Accept' answers instead of the 'lean towards' answers, the compatibilists are also much more certain in their belief, while the libertarians and no-free-willers tend to be uncertain much more often.

And the faculty is more definitely compatibilistic than the students, which seems to indicate education correlates with acceptance of compatibilism.

But more importantly: these people also seem to prefer to two-box in Newcomb's problem. So why should I put much weight in their opinion?

Comment author: Thrasymachus 21 September 2012 04:15:14PM 4 points [-]

A weak majority/strong plurality of relevant domain experts does not make the question decisively answered. I don't have survey data on this, but I'm pretty sure none of the compatibilists (even those who 'accept' it), take the question to be obviously answered etc. etc.

But more importantly: these people also seem to prefer to two-box in Newcomb's problem. So why should I put much weight in their opinion?

The majority of decision theory specialists two-box. I'm sure you can guess what I'm going to say about doman expertise and dunning-kruger effect here, too.

Comment author: ArisKatsaris 21 September 2012 04:24:30PM 1 point [-]

A weak majority/strong plurality of relevant domain experts does not make the question decisively answered.

Tell me, do you have any criterion over whether something is "decisively answered" other than how many "relevant domain experts" agree with it? If your definition of "decisively answered" is solely dependent on this, then we can just agree that we were using different definitions for the term.

The majority of decision theory specialists two-box.

So much for the decision theory specialists. Implement a real life version of Newcomb's box, where you fill in the opaque box based on whether they said they'll one-box or two-box. Assuming everyone follows what they said they should do, the one-boxers will just win, and the two-boxers will be weeping.

Comment author: Thrasymachus 22 September 2012 03:52:29PM 5 points [-]

Tell me, do you have any criterion over whether something is "decisively answered" other than how many "relevant domain experts" agree with it? If your definition of "decisively answered" is solely dependent on this, then we can just agree that we were using different definitions for the term.

I take 'decisively answered' to mean something along the lines of "here is an account, which, properly understood, solves this problem to the satisfaction of reasonable people". So (near) unanimity among relevant domain experts is necessary but not sufficient for this. I can't think of anything in natural language we would call a 'decisive answer' or similar in which 40% or so of relevant domain experts disagree with.

So much for the decision theory specialists. Implement a real life version of Newcomb's box, where you fill in the opaque box based on whether they said they'll one-box or two-box. Assuming everyone follows what they said they should do, the one-boxers will just win, and the two-boxers will be weeping.

This is recapitulating a standard argument for one-boxing, and it is well discussed in the literature. The fact the bulk of people who spend their time studying this issue and don't find this consideration decisive should make you think it is less a silver bullet than you think it is.

Comment author: ArisKatsaris 22 September 2012 11:20:22PM *  0 points [-]

This is recapitulating a standard argument for one-boxing, and it is well discussed in the literature. The fact the bulk of people who spend their time studying this issue and don't find this consideration decisive should make you think it is less a silver bullet than you think it is.

I should update slightly towards that direction, yes, but I have to note that the poll you gave me are not just about people who study the issue, but people who also seem to have made a career out of discussing it, and therefore (I would cynically suggest) perhaps wouldn't like the discussion to be definitively over.

e.g. Theologists and Priests are perhaps not the best people to poll, if you want to determine the existence of God.

Ah, but I just remembered atheism was one of the things you complained about being treated as obviously correct by most of us here? Because the domain experts about God (Theologists and Priests) haven't come to same conclusion?

This is recapitulating a standard argument for one-boxing,

I don't feel a pressing need to be non-standard: One-boxing wins, two-boxing loses -- that's all one needs to know for the purpose of choosing between them.

Comment author: Thrasymachus 24 September 2012 09:19:17PM 1 point [-]

I should update slightly towards that direction, yes, but I have to note that the poll you gave me are not just about people who study the issue, but people who also seem to have made a career out of discussing it, and therefore (I would cynically suggest) perhaps wouldn't like the discussion to be definitively over.

Sure, but I gather there are other things you can discuss in decision theory besides Newcomb's problem, so it isn't like the decision theorists need an artificial controversy about this to keep their jobs.

There are dissimilarities between decision theorists and (say) theologians, priests etc. Decision theorists are unlikely to have prior convictions about decision theory before starting to study it, unlike folks who discuss religion. The relevant domain expert in 'Does God exist' would likely be philosophers of religion, although there is a similar selection effect. However, for what it's worth, I doubt atheist philosophers of religion would consider the LW case for atheism remotely creditable.

Comment author: Peterdjones 21 September 2012 12:11:57PM *  3 points [-]

I don't think there's much value in pretending that issues like God (and the absence thereof) or the compatibility between determinism and (any logically coherent view of) free will haven't been decisively answered.

There are plenty of reasons for putting forward you conclusions as non decisive: (edited)

  1. Not sounding as though you are suffering from the Dunning Kruger effect

  2. Academic Modesty.

  3. You might actually be wrong. No one who calls themselves a rationalist should confuse "Seems true to me" with "is true".

Comment author: ArisKatsaris 21 September 2012 12:29:40PM 2 points [-]
  1. Not sounding as though you are suffering from the Dunning Kruger effect

  2. Academic Modesty.

Are those separate points?

  1. You might actually be wrong.

I 'might' also be wrong about the Earth not being flat. That still doesn't mean that we shouldn't consider the shape of the earth decisively answered.

Comment author: Peterdjones 21 September 2012 12:34:05PM 5 points [-]

They may overlap. Are they bad points?

I 'might' also be wrong about the Earth not being flat. That still doesn't mean that we shouldn't consider the shape of the earth decisively answered.

The pertinent point is that all informed opinion considers it decisiley answered. That is not the case with the two issues you cited as having been decisevly answered by EY.

Comment author: ArisKatsaris 21 September 2012 02:09:56PM *  2 points [-]

Are they bad points?

They're insufficient for me. Other people may find them sufficient.

The pertinent point is that all informed opinion considers it decisively answered

So, according to you, it seems I shouldn't pronounce something decisively answered unless "all informed opinion" considers it decisively answered.

Don't you see the paradox in this? How is the first person to consider it 'decisively answered' supposed to call it 'decisively answered', if he/she must first wait for all other people to call it 'decisively answered' first?

Comment author: Peterdjones 21 September 2012 02:42:40PM 4 points [-]

he/she must first wait for all other people to call it 'decisively answered' first?

No they needn't. They only need wait for the point to be reached where an overwhelming majority agree with an answer. Having noted that , they can correctly state that it has been decisevely answered. They only need others to agree with the anwer, not for others to agree that the question has been decisvely answered.

Comment author: ArisKatsaris 21 September 2012 03:02:31PM *  -1 points [-]

They only need wait for the point to be reached where an overwhelming majority agree with an answer

I don't think that "decisively answered" need have anything to do with democracy -- for example I'm sure that if you poll Czech scientists about the existence of God, you'll get a different distribution than if you ask Iranian scientists. Even if they're equally informed, political considerations will make them voice different things.

The policy you suggest seems designed to minimize conflict with your academic peers, not designed to maximize effectiveness in the pursuit of understanding the universe.

Comment author: Peterdjones 21 September 2012 03:34:15PM 0 points [-]

Churchill said democracy was the worst system apart from all the others. Do you have an alternative way of establishing Deciiveness that improves on the Majority of Informed Opinion?

'm sure that if you poll Czech scientists about the existence of God, you'll get a different distribution than if you ask Iranian scientists

Neither of those subsets would get me the majority of informed opinion. I believe I have already solved that problem.

Comment author: ArisKatsaris 21 September 2012 04:01:23PM *  0 points [-]

Churchill said democracy was the worst system apart from all the others.

Churchill's exact quote was "Democracy is the worst form of government, except for all those other forms that have been tried from time to time" He was talking about forms of government, not methods of understanding the universe.

Do you have an alternative way of establishing Deciiveness that improves on the Majority of Informed Opinion?

As a sidenote, let me note here that even on the issue you argued about, this "majority" seems to actually exist. The majority of philosophers are compatibilists, according to Thrasymachus's linked poll above.

And there seems to be an > 80% percentage (an overwhelming majority) against libertarian free will. According to your own argument then, even if you don't find compabilism a "decisive answer", you should find libertarianism a "decisive failure of an answer".

But getting back to your question: "Do you have an alternative way of establishing Deciiveness that improves on the Majority of Informed Opinion?"

Well, even if we don't speak about things like "Science" or "Testing" or "Occam's Razor properly utilized", I think I'll prefer the "Majority of Informed Opinion that Also Has IQ > 130 And Also One-Boxes in Newcomb's Dilemma".

Comment author: Morendil 15 September 2012 07:51:41AM 7 points [-]

I'd also mention

  • Hayakawa's Language In Thought and Action
  • Axelrod's The Evolution of Cooperation
  • Rawls' Theory of Justice (though the Sequences don't discuss CEV much)
  • maybe Peter Singer?
Comment author: Manfred 15 September 2012 08:09:00AM 6 points [-]

Could you connect them to the sequences like Luke did please? To the extent that I am familiar with your list, I'm having a hard time seeing it.

Comment author: Morendil 15 September 2012 08:59:19AM *  10 points [-]

Eliezer has explicitly mentioned Hayakawa in Intensions and Extensions. Axelrod is important to understanding the iterated Prisoner's Dilemma. Singer is one of the better-known proponents of a "shut up and multiply" approach to utilitarianism, agreeing with Eliezer's conclusions in various places (eg). Rawls' notions of the "veil of ignorance" and "reflective equilibrium" have been mentioned in connection with Coherent Extrapolated Volition - when I first came across CEV the similarities with Rawls stuck out like a sore thumb.

Comment author: Manfred 15 September 2012 09:15:21AM 1 point [-]

Thanks!

Comment author: lukeprog 15 September 2012 09:16:00PM *  5 points [-]

Hayakawa's Language In Thought and Action

...is not what I'd call "mainstream academia." Its program of "general semantics" is instead what Martin Gardner labeled as "cultism and pseudo-science" in one chapter of Fads and Fallacies in the Name of Science. Despite this, Language in Thought and Action is pretty good.

Comment author: John_Maxwell_IV 17 September 2012 01:26:22AM 3 points [-]

I edited the sequences page to add a link to this post.

Comment author: Eliezer_Yudkowsky 15 September 2012 02:04:28PM 3 points [-]

And lo, people began tweeting:

Eliezer Yudkowsky's "Sequences" are mostly not original

Which is false. This pushes as far in the opposite wrong direction as the viewpoint it means to criticize.

Evolutionary biology, the non-epistemological part of the exposition of quantum mechanics, and of course heuristics and biases, are all not original. They don't look deceptively original either; they cite or attributed-quote the sources from which they're taken. I have yet to encounter anyone who thinks the Sequences are more original than they are.

When it comes to the part that isn't reporting on standard science, the parts that are mostly dealt with by modern "philosophers" rather than experimental scientists of one kind or another, the OP is vastly overstating how much of the Sequences are similar to the standard stuff out there. There is such a vast variety of philosophy that you can often find a conclusion similar to anything, to around the same degree that Leibniz's monadology anticipated timeless quantum mechanics, i.e., not very much. The motivations, the arguments by which things are pinned down, the exact form of the conclusions, and what is done with those conclusions, is most of the substance - finding a conclusion that happens to look vaguely similar does not mean that I was reporting someone else's academic work and failing to cite it, or reinventing work that had already been done. It is not understating any sort of "close agreement" with even those particular concluders, let alone the field as a whole within which those are small isolated voices. Hofstadter's superrationality is an acknowledged informal forerunner of TDT. But finding other people who think you ought to cooperate in the PD, but can't quite formalize why, is not the same as TDT being preinvented. (Also TDT doesn't artifically sever decision nodes from anything upstream; the idea is that observing your algorithm, but not its output, is supposed to screen off things upstream. This is "similar" to some attempts to rescue evidential decision theory by e.g. Eels, but not quite the same thing when it comes to important details like not two-boxing on Newcomb's Problem.) And claiming that in principle philosophical intuitions arise within the brain is not the same as performing any particular dissolution of a confused question, or even the general methodology of dissolution as practiced and described by Yudkowsky or Drescher (who actually does agree and demonstrate the method in detail within "Good and Real").

I'm also still not sure that Luke quite understands what the metaethics sequence is trying to say, but then I consider that sequence to have basically failed at exposition anyway. Unfortunately, there's nothing I can point Luke or anyone else at which says the same thing in more academic language.

Several of these citations are from after the originals were written! Why not (falsely) claim that academia is just agreeing with the Sequences, instead?

I don't understand what the purpose of this post was supposed to be - what positive consequence it was supposed to have. Lots of the Sequences are better exposition of existing ideas about evolutionary biology or cognitive biases or probability theory or whatever, which are appropriately quoted or cited within them? Yes, they are. People introducing Less Wrong should try to refer to those sources as much as possible when it comes to things like heuristics and biases, rather than talking like Eliezer Yudkowsky somehow invented the idea of scope insensitivity, so that they don't sound like phyg victims? Double yes. But writing something that predictably causes some readers to get the impression that ideas presented within the Sequences are just redoing the work of other academics, so that they predictably tweet,

Eliezer Yudkowsky's "Sequences" are mostly not original

...I do not think the creation of this misunderstanding benefits anyone. It is also a grave sin to make it sound like you're speaking for a standard academic position when you're not!

And I think Luke is being extremely charitable in his construal of what's "already" been done in academia. If some future anti-Luke is this charitable in construing how much of future work in epistemology and decision theory was "really" all done within the Sequences back in 2008, they will claim that everything was just invented by Eliezer Yudkowsky way back then - and they will be wrong - and I hope somebody argues with that anti-Luke too, and doesn't let any good feeling for ol E. Y. stand in their way, just like we shouldn't be prejudiced here by wanting to affiliate with academia or something.

I get what this is trying to do. There's a spirit in LW which really is a spirit that exists in many other places, you can get it from Feynman, Hofstadter, the better class of science fiction, Tooby and Cosmides, many beautiful papers that were truly written to explain things as simply as possible, the same place I got it. (Interesting side note: John Tooby is apparently an SF fan who grew up reading van Vogt and Null-A, so he got some of his spirit from the same sources I did! There really is an ancient and honorable tradition out there.) If someone encounters that spirit in LW for the first time, they'll think I invented it. Which I most certainly did not. If LW is your first introduction to these things, then you really aren't going to know how much of the spirit I learned from the anncient masters... because just reading a citation, or even a paragraph-long quote, isn't going to convey that at all. The only real way for people to learn better is to go out and read Language in Thought and Action or The Psychological Foundations of Culture. Doing this, I would guess, gave Luke an epiphany he's trying to share - there's a whole world out there, not just LW the way I first thought. But the OP doesn't do that. It doesn't get people to read the literature. Why should they? From what they can see, it's already been presented to them on LW, after all. So they won't actually read the literature and find out for themselves that it's not what they've already read.

There's literature out there which is written in the same spirit as LW, but with different content. Now that's an exciting message. It might even get people to read things.

Comment author: lukeprog 15 September 2012 10:35:41PM *  55 points [-]

the OP is vastly overstating how much of the Sequences are similar to the standard stuff out there... I think Luke is being extremely charitable in his construal of what's "already" been done in academia

Do you have a Greasemonkey script that rips all the qualifying words out of my post, or something? I said things like:

  • "Eliezer's posts on evolution mostly cover material you can find in any good evolutionary biology textbook"
  • "much of the Quantum Physics sequence can be found in quantum physics textbooks"
  • "Eliezer's metaethics sequences includes dozens of lemmas previously discussed by philosophers"
  • "Eliezer's free will mini-sequence includes coverage of topics not usually mentioned when philosophers discuss free will (e.g. Judea Pearl's work on causality), but the conclusion is standard compatibilism."
  • "[Eliezer's posts] suggest that many philosophical problems can be dissolved into inquiries into the cognitive mechanisms that produce them, as also discussed in"
  • "[Eliezer's posts] make the point that value is complex, a topic explored in more detail in..."

Your comment above seems to be reacting to a different post that I didn't write, one that includes (false) claims like: "The motivations, the arguments by which things are pinned down, the exact form of the conclusions are mostly the same between The Sequences and previous work in mainstream academia."

I have yet to encounter anyone who thinks the Sequences are more original than they are.

Really? This is the default reaction I encounter. Notice that when the user 'Thomas' below tried to name just two things he thought were original with you, he got both of them wrong.

Here's a report of my experiences:

  • People have been talking about TDT for years but nobody seems to have noticed Spohn until HamletHenna and I independently stumbled on him this summer.

  • I do find it hard to interpret the metaethics sequence, so I'm not sure I grok everything you're trying to say there. Maybe you can explain it to me sometime. In any case, when it comes to the pieces of it that can be found elsewhere, I almost never encounter anyone who knows their earlier counterparts in (e.g.) Railton & Jackson — unless I'm speaking to someone who has studied metaethics before, like Carl.

  • A sizable minority of people I talk to about dissolving questions are familiar with the logical positivists, but almost none of them are familiar with the recent cogsci-informed stuff, like Shafir (1998) or Talbot (2009).

  • As I recall, Less Wrong had never mentioned the field of "Bayesian epistemology" until my first post, The Neglected Virtue of Scholarship.

  • Here's a specific story. I once told Anna that once I read about intelligence explosion I understood right away that it would be disastrous by default, because human values are incredibly complex. She seemed surprised and a bit suspicious and said "Why, had you read Joshua Greene?" I said "Sure, but he's just one tip of a very large iceberg of philosophical and scientific work demonstrating the complexity of value. I was convinced of the complexity of value long ago by metaethics and moral psychology in general."

Several of these citations are from after the originals were written! Why not (falsely) claim that academia is just agreeing with the Sequences, instead?

Let's look at them more closely:

  • Lots of cited textbooks were written after the Sequences, because I wanted to point people to up-to-date sources, but of course they mostly summarize results that are a decade old or older. This includes books like Glimcher (2010) and Dolan & Sharot (2011).

  • Batson (2011) is a summary of Batson's life's work on altruism in humans, almost all of which was published prior to the Sequences.

  • Spohn (2012) is just an update to Spohn's pre-Sequences on work on his TDT-ish decision theory, included for completeness.

  • Talbot (2009) is the only one I see that is almost entirely composed of content that originates after the Sequences, and it too was included for completeness immediately after another work written before the Sequences: Sharif (1998).

I don't understand what the purpose of this post was supposed to be - what positive consequence it was supposed to have.

That's too bad, since I answered this question at the top of the post. I am trying to counteract these three effects:

  1. Some readers will mistakenly think that common Less Wrong views are more parochial than they really are.
  2. Some readers will mistakenly think Eliezer's Sequences are more original than they really are.
  3. If readers want to know more about the topic of a given article, it will be more difficult for them to find the related works in academia than if those works had been cited in Eliezer's article.

I find problem #1 to be very common, and a contributor to the harmful, false, and popular idea that Less Wrong is a phyg. I've been in many conversations in which (1) someone starts out talking as though Less Wrong views are parochial and weird, and then (2) I explain the mainstream work behind or similar to every point they raise as parochial and weird, and then (3) after this happens 5 times in a row they seem kind of embarrassed and try to pretend like they never said things suggesting that Less Wrong views are parochial and weird, and ask me to email them some non-LW works on these subjects.

Problem #2 is common (see the first part of this comment), and seems to lead to phygish hero worship, as has been pointed out before.

Problem #3, I should think, is uncontroversial. Many of your posts have citations to related work, most of them do not (as is standard practice in the blogosphere), and like I said I don't think it would have been a good idea for you to spend time digging up citations instead of writing the next blog post.

writing something that predictably causes some readers to get the impression that ideas presented within the Sequences are just redoing the work of other academics, so that they predictably tweet ...I do not think the creation of this misunderstanding benefits anyone

Predictable misunderstandings are the default outcome of almost anything 100+ people read. There's always a trade-off between maximal clarity, readability, and other factors. But, I'm happy to tweak my original post to try to counteract this specific misunderstanding. I've added the line: "(edit: probably most of their content is original)".

[Further reading, I would guess] gave Luke an epiphany he's trying to share - there's a whole world out there, not just LW the way I first thought.

Remember that I came to LW with a philosophy and cogsci (especially rationality) background, and had been blogging about biases and metaethics and probability theory and so on at CommonSenseAtheism.com for years prior to encountering LW.

I get what this is trying to do. There's a spirit in LW which really is a spirit that exists in many other places, you can get it from Feynman, Hofstadter, the better class of science fiction, Tooby and Cosmides, many beautiful papers that were truly written to explain things as simply as possible, the same place I got it.

That is definitely not the spirit of my post. If you'll recall, I once told you that if all human writing were about to be destroyed except for one book of our choosing, I'd go with The Sequences. You can't get the kind of thing that CFAR is doing solely from Feynman, Kahneman, Stanovich, etc. And you can't get FAI solely from Good, Minsky, and Wallach — not even close. Again, I get the sense you're reacting to a post with different phrasing than the one I actually wrote.

So they won't actually read the literature and find out for themselves that it's not what they've already read.

Most people won't read the literature either you or I link to. But many people will, like Wei Dai.

Case in point: Remember Benja's recent post on UDT that you praised as "Original scientific research on saving the world"? Benja himself wrote that the idea for that post clicked for him as a result of reading one of the papers on logical uncertainty I linked to from So You Want to Save the World.

Most people won't read my references. But some of those who do will go on to make a sizable difference as a result. And that is one of the reasons I cite so many related works, even if they're not perfectly identical to the thing me or somebody else is doing.

Comment author: Wei_Dai 17 September 2012 12:06:04PM *  9 points [-]

I am trying to counteract these three effects

I think a valid criticism can be made that while you were trying to counteract these three effects (which is clearly an important and useful effort), you didn't take enough care to avoid introducing a new effect, of making some people think the Sequences are less original than they actually are. (For example you didn't ask Eliezer to double check your descriptions of how the Sequences posts relate to the academic works, and you didn't give some examples of where the Sequences are original.)

This is bad because in addition to communicating various ideas, the Sequences also serve as evidence of Eliezer's philosophy and rationality talents/skills, which is useful for potential donors/supporters to judge the likely future effectiveness of the Singularity Institute in achieving its goals.

Comment author: lukeprog 18 September 2012 02:54:50AM 4 points [-]

I agree I could have spent a paragraph reinforcing the originality of The Sequences.

As for asking Eliezer to check the article before posting: I've sent Eliezer things for feedback before, and he usually doesn't give feedback on them until after I stop waiting and post them to LW. But as a result of this post, we've arranged a new heuristic: If I think Eliezer plausibly disagrees with a thing I'm going to post to LW, I'll give him a chance to give feedback on it before I post it.

Comment author: TimS 17 September 2012 01:17:36PM *  2 points [-]

From a donor point of view, the question is as much whether Eliezer has made relevant lessons a true part of him as whether he has done original work.

The Sequences are neither necessary nor sufficient to get funding to do actual research (although I hope they are helpful in obtaining funding for research).

Comment author: [deleted] 16 September 2012 04:14:59PM *  17 points [-]

Most people won't read my references. But some of those who do will go on to make a sizable difference as a result. And that is one of the reasons I cite so many related works, even if they're not perfectly identical to the thing me or somebody else is doing.

FWIW, Luke's rigorous citation of references has been absurdly useful to me when doing my research. It's one of the aspects of reading LW that makes it worthwhile and productive.

Luke is already aware that I've utilized his citations to a great extent, but I wanted to publicly thank him for all that awesome work. I'd also like to thank others who have done similar things, such as Klevador. We need more of this.

Comment author: CarlShulman 16 September 2012 07:33:34AM *  4 points [-]

On complexity of value, I didn't see anyone talking about the details of neuroeconomics nor the neuroscientific distinction between "pleasure" and "desire" until I started posting about them

Yvain has posted more than once on this, although with less detail and referencing.

Comment author: lukeprog 16 September 2012 07:36:55AM *  6 points [-]

Oops, fixed. Thanks.

Though, note that the second Yvain post you linked to was a follow-up to one of my reference-packed posts on the subject.

Comment author: Eliezer_Yudkowsky 16 September 2012 04:18:49AM 3 points [-]

Do you have a Greasemonkey script that rips all the qualifying words out of my post, or something?

All readers have a Greasemonkey script that rips all the qualifying words out of a post. This is a natural fact of writing and reading.

Your comment above seems to be reacting to a different post that I didn't write

Not the post you wrote - the post that the long-time LWer who Twittered "Eliezer's Yudkowsky's Sequences are mostly not original" read. The actual real-world consequences of a post like this when people actually read it are what bothers me, and it does feel frustrating because those consequences seem very predictable - like you're living in an authorial should-universe. Of course somebody's going to read that post and think "Eliezer Yudkowsky's Sequences are mostly not original"! Of course that's going to be the consequence of writing it! And maybe it's just because I was reading it instead of writing it myself, without having all of your intentions so prominently in my mind, but I don't see why on Earth you'd expect any other message to come across than that. A few qualifying words don't have the kind of power it takes to stop that from happening!

Comment author: lukeprog 16 September 2012 04:58:37AM *  28 points [-]

All readers have a Greasemonkey script that rips all the qualifying words out of a post... I don't see why on Earth you'd expect any other message to come across than ["Eliezer's Sequences are mostly not original"].

Do you think most readers misinterpreted my post in that way? I doubt it. It looks to me like one person tweeted "Eliezer's Sequences mostly not original" — a misinterpretation of my post which I've now explicitly denied near the top of the post.

My guess now would be that I probably underestimate the degree to which readers misinterpreted my post (because my own intentions were clear in my mind, illusion of transparency), and that you probably overestimate the degree to which readers misinterpreted my post (because you seem to have initially misinterpreted it, and that misinterpretation diminishes several years of cognitive work that you are justly proud of).

Also: you seem to be focusing on the one tweeted misinterpretation and not taking into account that we have evidence that the post is also achieving its explicitly stated goals, as evidenced by many of the comments on this thread: 1, 2, 3, 4, 5.

Comment author: TimS 16 September 2012 04:54:37AM *  9 points [-]

It is very easy to read the sequences and think that you think the philosophical thought is original to you. Other than the FAI stuff and decision theory stuff, is that true?

What exactly is wrong with being thought of as a very high-end popularizer? That material is incredibly well presented.

Additionally, people who disagree with your philosophical positions ought not be put in the (EDIT: position) of needing to reinvent the philosophical wheel to engage critically with your essays.

Comment author: wedrifid 16 September 2012 05:00:02AM 0 points [-]

Additionally, people who disagree with your philosophical positions ought not be put in the power of needing to reinvent the philosophical wheel to engage critically with your essays.

Put in the position of?

Comment author: TimS 16 September 2012 03:58:29PM 0 points [-]

Yes, thanks.

Comment author: ciphergoth 18 September 2012 12:58:19PM 2 points [-]

I'd take out the EDIT - people can see from the comment below that you edited in response to a comment.

Comment author: army1987 16 September 2012 06:44:51PM 7 points [-]

All readers have a Greasemonkey script that rips all the qualifying words out of a post. This is a natural fact of writing and reading.

I don't. In fact, I sometimes insert such words.

Comment author: [deleted] 21 September 2012 07:45:14AM 3 points [-]

Of course somebody's going to read that post and think "Eliezer Yudkowsky's Sequences are mostly not original"! Of course that's going to be the consequence of writing it!

Only a single conclusion is possible: LukeProg is a TRAITOR!

Comment author: wedrifid 21 September 2012 11:13:19AM 8 points [-]

Only a single conclusion is possible: LukeProg is a TRAITOR!

I can understand why this would be negatively received by some---it is clearly hyperbole with a degree of silliness involved. That said---and possibly coincidentally---there is a serious point here. In fact it is the most salient point I noticed when reading the post and initial responses.

In most social hierarchies this post would be seen as a betrayal. An unusually overt and public political move against Eliezer. Not necessarily treason, betrayal of the tribe, it is a move against a rival. Of course it would certainly be in the interest of the targeted rival to try to portray the move as treason (or heresy, or whatever other kind of betrayal of the tribe rather than mere personal conflict.)

The above consideration is why I initially expected Eliezer to agree to a larger extent than he did (which evidently wasn't very much!) Before making public statements of a highly status sensitive nature regarding an ally the typical political actor will make sure they aren't offending them---they don't take the small risk establishing an active rivalry unless they are certain the payoffs are worth it.

This (definitely!) isn't to say that any of the above applies to this situation. Rationalists are weird and in particular can have an unusual relationship between their intellectual and political expression. ie. They sometimes go around saying what they think.

Comment author: Eliezer_Yudkowsky 21 September 2012 01:45:59PM 8 points [-]

The thought that Luke was trying to sabotage my position, consciously or unconsciously, honestly never crossed my mind until I read this comment. Having now considered the hypothesis rather briefly, I assign it a rather low probability. Luke's not like that.

Comment author: TheOtherDave 21 September 2012 07:37:47PM 2 points [-]

It is perhaps worth noting that wedrifid didn't say anything about motives (conscious or otherwise).

Whether I believe someone is trying to sabotage my position (consciously or unconsciously) is a different question from whether I believe they are making a move against me in a shared social hierarchy. (Although each is evidence for the other, of course.)

Comment author: lukeprog 16 September 2012 07:19:39AM *  12 points [-]

There's literature out there which is written in the same spirit as LW, but with different content. Now that's an exciting message. It might even get people to read things.

Maybe we can start to build up a repository of those things, too. So far, you've recommended:

  • Language in Thought and Action
  • Psychological Foundations of Culture
  • Good and Real
  • Rational Choice in an Uncertain World

Unfortunately, those works seem incredibly different to me, so it's hard for me to guess which other works you would also endorse as being in the "LW spirit." I'll try anyway:

Comment author: pjeby 17 September 2012 07:25:07PM 5 points [-]

How about:

Written by a psychologist-philosopher (literally), it reads exactly like a Sequence on five-second approaches to a wide array of thinking errors, carefully cataloged and taxonomized with the information needed to get out of them... and most of them are not thinking errors that have previously been cataloged on LW.

(Even what we commonly refer to here under the heading of "sunk-cost fallacy" is given a much more rigorous, "five-second level" analysis, showing how we get stuck in that fallacy all day long doing ordinary things. Forget sticking with a big multi-year project, he shows how we can get skewered by this fallacy in doing things that take five minutes.)

Comment author: curiousepic 19 September 2012 05:56:23PM *  0 points [-]

Written by a psychologist-philosopher (literally)

There should be more (literal) philosopher-psychologists.

Comment author: Randaly 22 September 2012 02:27:44AM 3 points [-]
Comment author: buybuydandavis 15 September 2012 08:48:51PM 12 points [-]

There's a spirit in LW which really is a spirit that exists in many other places,

Yes, and pointing out those other places here serves two purposes.

It serves to brand LW, so that people passing by can quickly see the kind of spirit here. Yes, there's a whole world out there, and many of us have spent some time in it, so seeing references to that world here serves to quickly communicate some of what LW is about.

References also server to point people here to other expositions of similar material.

For example, you say:

The only real way for people to learn better is to go out and read Language in Thought and Action

I'd recommend people at some point move on from Hayakawa to Korzybski, Science and Sanity, and the whole General Semantics literature. People have spent decades discussing these issues and organizing

It doesn't get people to read the literature. Why should they? From what they can see, it's already been presented to them on LW, after all.

That's not my reaction to references. When I first came here, the references to Jaynes didn't make me think "I've already covered this stuff, no need to read this web site." On the contrary, it made me want to read more. Similarly, seeing a reference to other work associated with a sequence wouldn't make me think "no need to bother reading that, EY has already regurgitated it for me", it would make me want to read the original.

Yes, there's a whole world out there, making it easier to navigate that world with links is a good thing.

Comment author: lukeprog 15 September 2012 11:15:10PM *  11 points [-]

TDT doesn't artifically sever decision nodes from anything upstream; the idea is that observing your algorithm, but not its output, is supposed to screen off things upstream.

Pardon me; I'm not yet much of an expert with LW decision theories. When you explained TDT on the whiteboard to Alex (with me listening), you kept talking about "severing" rather than "screening off." I'll try to find a way to modify the OP.

ETA: I remembered I have a recording of that tutorial, and I when checked the recording, and it turns out my memory was wrong. You did talk about how TDT "screened off" the information whereas CDT "severs" the causal diagram.

Comment author: wedrifid 15 September 2012 03:53:21PM 8 points [-]

Thankyou for clearing that up. Given your occupational affiliation with Luke I had been overestimating the extent to which you would endorse his position. That is, I wouldn't have expected Luke to write this without checking with you first so thought you must have agreed.

Comment author: Morendil 15 September 2012 03:07:06PM *  12 points [-]

I don't understand what the purpose of this post was supposed to be - what positive consequence it was supposed to have.

I took the post to be Luke writing notes to himself, in public so as to recruit others' help, toward the kind of bibliography that might be included in an academically acceptable version of the Sequences, or of some parts of them.

The intention being, I gathered, to publish these bibliographies as an adjunct to the Sequences - perhaps in the "wall of references" style of Luke's early posts. (If so, I hope a more user-friendly way of displaying those is worked out first!)

(ETA: the specific positive consequence of that would be to help the reader "find the related works in academia" as per Luke's third numbered point in the OP.)

Comment author: Eliezer_Yudkowsky 15 September 2012 03:40:37PM -1 points [-]

Why would that actually be a consequence of the OP as written?

Comment author: Morendil 15 September 2012 04:37:06PM 7 points [-]

I'm stating what I discerned of the intention - I won't presume to judge the OP either as a plan of action, or as a first step in its execution.

Completely agree with your latter addendum that people should read Hofstadter, Hayakawa etc. not as footnotes to your work but for their own merits. Hofstadter I discovered in childhood and I wouldn't be the same person if I hadn't; I read Hayakawa on your recommendation, and am glad I did. Yay to more discussion of works that have the LW-nature, but are not otherwise alluded to in the Sequences. :)

Comment author: ciphergoth 16 September 2012 10:01:46AM 18 points [-]

I hadn't expected you to disagree with that tweet, so I'm clearly getting something wrong. I wrote that in the hope that it would encourage people to read the Sequences, not put them off - I think people imagine it as this million-word work of revelation, but a very large part of what it is is a work of popular science - turning people on to good existing ideas in psychology and philosophy and biology and physics and suchlike. There is a great deal that is original and valuable in there, but I don't think of it as the majority of the material.

Comment author: army1987 16 September 2012 06:42:09PM 23 points [-]

I get your point, but to lots of people the wording of that tweet would have the connotation ‘EY is a plagiarist’, not ‘EY is not a crackpot’.

Comment author: CronoDAS 17 September 2012 04:08:18AM 0 points [-]

Yes, this.

Comment author: ciphergoth 16 September 2012 10:40:10AM 7 points [-]

Thinking about it further though, this makes something of a nonsense of the original tweet, since it's hard to think what would count as "mostly original" by this standard. You might as well describe eg The Better Angels of Our Nature as "mostly not original" since it contains no original research but presents a synthesis of the research of others, building up to a common theme.

The problem I have is that if I say something that sounds positive about the Sequences, that's going to turn my friends off, since they already know I think well of them. By saying something that on first reading sounds negative, I might get their interest, but that only works if they go on to follow the link.

Comment author: wedrifid 16 September 2012 10:53:17AM 6 points [-]

The problem I have is that if I say something that sounds positive about the Sequences, that's going to turn my friends off, since they already know I think well of them.

For example, they may be turned off if you came out and said "The sequences really aren't the parochial ramblings of an intellectual outcast, they are totally in accord with mainstream scientific thinking". But "mostly not original" conveys much of the same message by making a concession to the orthodoxy.

Comment author: RichardKennaway 16 September 2012 11:20:48AM 0 points [-]

The problem I have is that if I say something that sounds positive about the Sequences, that's going to turn my friends off, since they already know I think well of them.

I do not understand this. What planet are your friends from? If you're tweeting to your friends, and they already know what you think of the Sequences, why are you tweeting about them to them?

Comment author: ciphergoth 16 September 2012 11:34:26AM 3 points [-]

They are from Earth. Because it would be great for me and for the world if more of my friends took an interest in this sort of thing, and if they have misconceptions that stand in the way of that I'd like to clear up those misconceptions.

Comment author: RichardKennaway 17 September 2012 08:01:17AM 1 point [-]

Because it would be great for me and for the world if more of my friends took an interest in this sort of thing, and if they have misconceptions that stand in the way of that I'd like to clear up those misconceptions.

I understand the goal; but not the action taken to achieve it. Negging the Sequences will get them to take more of an interest?

Comment author: ArisKatsaris 18 September 2012 09:25:27AM *  8 points [-]

The word "original" has positive connotations. And therefore the words "unoriginal" or "not original" have negative connotations.

So, yeah, I don't think you'd encourage anyone to read anything by calling it "not original".

Comment author: army1987 18 September 2012 05:45:02PM 3 points [-]

The word "original" has positive connotations.

Except on Wikipedia (where it's usually an euphemism for ‘crackpottish’). ;-)

(As someone on a Wikipedia talk page once said -- quoting from memory, “if we aren't allowed to [do X] the allowed band between original research and plagiarism becomes dangerously narrow”.)

Comment author: RichardKennaway 18 September 2012 10:46:04AM *  2 points [-]

So, yeah, I don't think you'd encourage anyone to read anything by calling it "not original".

I don't know anything about the friends ciphergoth is attempting to reach, but I observe that in religion, "original" would be the greater turn-off. In religion, every innovation is heretical, because it is an innovation. To be accepted it must be presented as "not original", either because it is exactly in accordance with official doctrine, or because it is a return to the true religion that the official doctrine has departed from. It is rare for a religion to successfully introduce a new prophet with the power to sweep away the old, and even then ("I am not come to destroy, but to fulfil") the pretence is maintained that no such thing has happened.

Comment author: ryjm 18 September 2012 02:20:30PM 0 points [-]

Someone who doesn't want to read science-y stuff because they have that kind of mindset is not going to suddenly become curious when someone tells them it's based on science-y stuff from less than 30 years ago.

I like to think of it temporally; that religion is much like rationalists facing the wrong direction. Both occasionally look over their shoulders to confirm their beliefs (although with theists it's more like throwing a homunculus into the distant past and using that for eyes), while most of the time the things we really care about and find exciting are in front of us. Original vs unoriginal with respect to modern thought is of no practical interest to someone with the "every innovation is heretical" mindset unless it is completely within their usual line of sight - heretical is code for "I don't want to keep looking over my shoulder", not "I hate the original on principle". So unless you put that "original" encouragement thousands of years ago where they can see it, where it's a matter of one in front and one behind, the distinction between which is the greater turn-off is not going to matter, or bait anyone into turning around - there is nothing in their usually observed world to relate it to.

Comment author: ciphergoth 18 September 2012 12:24:02PM 1 point [-]

Right, but I had hoped that the result would be that someone would follow the link in the tweet, after which they find out some things that may cause them to feel more positively.

Comment author: chaosmosis 16 September 2012 04:04:53AM *  29 points [-]

With both your comment here and your comments on the troll-fee issue I've found you coming across as arrogant. This perception seems to roughly match the response that other people have had to those comments as well, since most people disagreed with you in both areas (judging by number of upvotes). I hadn't perceived you that way before now, so I'm wondering if something happened to you recently that's altered the way you post or the way you think. This change is for the worse; I want my old model of Eliezer Yudkowsky back!

Frankly, I have found the sequences to be primarily useful for condensing concepts that I already had inside my head. The ideas expressed in almost all of the sequences are blatantly obvious, but they come across as catchy and often are reducible to a quick phrase. Their value lies in the fact that they make it easy to internalize certain ideas so that they're more readily accessible to me. They also helped clarify the boundaries of some concepts, to a certain extent. The sequences have provided me with a useful terminology, but I don't think they've offered me much else.

What ideas do you believe to be original that you've produced?

Is there a reason that defending the originality of the sequences is so important to you?

Comment author: IlyaShpitser 18 September 2012 12:22:40AM 7 points [-]

With both your comment here and your comments on the troll-fee issue I've found you coming across as arrogant.

You only got this now?

Comment author: Randaly 22 September 2012 10:52:07PM 0 points [-]

While it wasn't perfectly phrased, I understand where chaosmosis is coming from: I too get the sense that Eliezer is responding significantly less well to criticism, both by misinterpreting or straw-manning what other people have written and letting negative emotions influence what he writes. However, I don't think that one draw a line through two data points: after all, what I regard as Eliezer's best response to criticism, Reply to Holden on 'Tool AI', was written well after the Sequences.

Comment author: BayesLives 16 September 2012 03:38:16PM 7 points [-]

"Is there a reason that defending the originality of the sequences is so important to you?"

Yudkowsky may need to begin reviewing the literature on cognitive biases for his own sake at this point.

Comment author: atorm 17 September 2012 04:59:06AM 1 point [-]

I want my old model of Eliezer Yudkowsky back!

Eliezer Yudkowsky is the supreme being to whom it is up to all of us to become superior!

Comment author: wedrifid 17 September 2012 05:48:09AM *  0 points [-]

I want my old model of Eliezer Yudkowsky back!

Eliezer Yudkowsky is the supreme being to whom it is up to all of us to become superior!

I think chaosmosis would prefer to perceive this as occurring through a change in chaosmosis than a change in chaosmosis's evidence about Eliezer.

Comment author: chaosmosis 17 September 2012 01:09:31PM 0 points [-]

No preference.

I don't understand how your comment is responsive to atorm's though, so I might be missing something here.

Comment author: wedrifid 17 September 2012 01:42:49PM *  0 points [-]

I don't understand how your comment is responsive to atorm's though, so I might be missing something here.

It responds to the disconnect between the quote and the quoted quote, in particular the implication of the latter regarding the former.

Comment author: Thrasymachus 18 September 2012 08:30:55AM *  14 points [-]

One anecdote given the 'PR' worries raised:

I have never read the sequences. After reading Luke's post, I am much less likely to: the impression given is the sequences are generally idiosyncratic takes which recapitulate an already existing and better organized literature. I also think it is more likely the sequences are overrated, either through readers being unaware their (or similar) insights have already been made, or lacking the technical background to critique them.

It also downgraded my estimate of the value of EY's work. Although I was pretty sceptical, I knew there was at least some chance that the sequences really were bursting with new insights and that LW really was streets ahead of mainstream academia. This now seems much less likely - although I don't think EY is a plagiarist, it seems most of the sequences aren't breaking new ground, but summarizing/unwittingly recapitulating insights that have already been made and taken further elsewhere.

So I can see the motivation for EY to defend that their originality: his stock goes down if the sequences are neat summaries but nothing that new rather than bursting with new and important insights, and EY's stock is important for things like donations, public perception of him and the SI, etc. (Both my likelihood of donating and my regard for SI has been lowered a bit by this post and comments). However, EY's way of responding to (weakly implied) criticism with catty arrogance compounds the harm.

Comment author: Yvain 18 September 2012 10:28:23AM *  34 points [-]

If you are at all interested in rationality it would be a huge shame for you to skip the Sequences.

Yes, a lot of the material in the Sequences could also be obtained by reading very very carefully a few hundred impenetrable scholarly books that most people have never heard of in five or ten different disciplines, supplemented by a few journal articles, plus some additional insights by "reading between the lines", plus drawing all the necessary connections between them. But you will not do this.

The Sequences condense all that information, put it in a really fun, really fascinating format, and transfer all of it into the deepest levels of your brain in a way that those hundred books wouldn't. And then there's some really valuable new material. Luke and Eliezer can argue whether the new material is 30% of the Sequences or 60% of the Sequences, but either number is still way more output than most people will produce over their entire lives.

If your worry is that they will just be recapitulating things you already know, I am pretty doubtful; I don't know your exact knowledge level, but they were pretty exciting for me when I first read them and I had college degrees in philosophy and psychology which are pretty much the subjects covered. And if they are new to you, then from a "whether you should read them" point of view it doesn't matter if Eliezer copied them verbatim off Wikipedia.

Seriously. Read the Sequences. Luke, who is the one arguing against their originality above, says that they are the one book he would like to save if there was an apocalypse. I would have to think a long time before saying the same but they're certainly up there.

Also, as a fellow doctor interested in utiltiarianism/efficient charity, I enjoyed your blog and associated links.

Comment author: lukeprog 23 September 2012 06:21:11PM 4 points [-]

Luke and Eliezer can argue whether the new material is 30% of the Sequences or 60% of the Sequences...

For the record, when I read Eliezer's comments about the originality of The Sequences, it sounds to me like he and I have pretty much the same estimate of how original The Sequences are.

Comment author: army1987 18 September 2012 05:37:52PM 3 points [-]

If you are at all interested in rationality it would be a huge shame for you to skip the Sequences.

You might want to link to "Yes, a blog" by Academian.

Comment author: Thrasymachus 21 September 2012 03:08:35PM 2 points [-]

Fair enough. Your and Luke's recommendation are enough for me to read at least some to see if I have got the wrong impression.

Comment author: Epiphany 21 September 2012 04:25:26AM 2 points [-]

The sequences need a summary like the one you just wrote, the way books have a summary on the cover. Maybe this should be taken as a hint that you'd get more mileage out of the sequences with a really good description placed prominently in front of them. That could quickly re-frame non-originality claims as being irrelevant by plainly stating that they're an accessible and entertaining way to learn about logic and bias (implying that the presentation is valuable even if some of the content can be found elsewhere), with (whatever amount) of new content on X, Y, Z topics. If you choose to write such a description, I'd really like to know what you got out of them that your philosophy and psychology degrees didn't give you.

Comment author: paper-machine 21 September 2012 04:49:57AM 1 point [-]

The sequences need a second edition. It's sheer hubris to think that nothing has changed in four years.

Comment author: wedrifid 21 September 2012 11:40:14AM 6 points [-]

The sequences need a second edition. It's sheer hubris to think that nothing has changed in four years.

There would be room for improvement even without anything changing. They were produced as daily blog posts for the purpose of forcing Eliezer to get his thoughts down on a page.

Comment author: Peterdjones 21 September 2012 05:04:47PM *  5 points [-]

Actually I think the sequences are worth reading even though I deplore the tub-thumping, lack of informedness, etc.

What would you expect if someone bright but uninformed about philosophy invented their own philosophy?

Lots of ground re-covered. Lots of avoidable errors. Some novel insights.

Comment deleted 05 October 2012 04:59:06PM *  [-]
Comment author: wedrifid 05 October 2012 05:43:06PM *  -2 points [-]

The sheer magnitude is what impresses gullible readers of the Sequences.

Wow. Deja vu. I actually have to follow this link and double check the date to see if this was the same comment we dealt with before or just a repetition of the same agenda by the same sockpuppet. If you check DevilWorm's user page you will see that this comment is a copy and paste clone of one he previously made that has now been deleted or banned (5 comments below on that page, to be precise). Once again it has received initial upvotes---either from his other accounts or from users who are vulnerable to persuasion on DevilWorm's only topic of discussion (the worthlessness of Eliezer Yudkowsky).

Come on, when I want to harp on about one issue repetitively I at least either make up new speech every time or make an explicit link to the previous one.

Comment deleted 05 October 2012 05:46:57PM [-]
Comment author: wedrifid 05 October 2012 05:56:51PM *  2 points [-]

And why was it deleted, when I've posted far more "objectionable" matter?

Because the moderators don't have access to a "ban account' feature for accounts that only post 'objectionable' material.

Comment deleted 05 October 2012 06:09:21PM *  [-]
Comment author: wedrifid 05 October 2012 06:15:29PM *  3 points [-]

So, you approve of the practice of disappearing comments without any notice of the fact or the reason?

Not as such, but I approve of disappearing anything everything from known trolling sockpuppet accounts.

(I feel like I should be paying a 5 karma troll-feeding-toll to write this but for some reason there are upvotes where I expected downvotes. I'll wait a day to see how things stabilize then consider if my model of lesswrong users needs to be updated.)

Comment author: pragmatist 05 October 2012 05:30:23PM 0 points [-]

That's why he always refuses to summarize his conclusions.

It seems like his latest sequence is offering summarized versions of at least some of the previous sequences.

Comment author: Wei_Dai 15 September 2012 07:41:25PM *  14 points [-]

I agree that Luke's post might cause some people to update too much in the direction of "the Sequences aren't original". He was wrong or overstated things in the couple of bullet points that I checked out (and pointed out in my earlier comment). He probably should have showed it to you for error-checking and making sure it's being fair before posting it.

I do think having an index of related works is very valuable, for people wanting to do further readings, or figuring out exactly which parts of the Sequences are original.

So they won't actually read the literature and find out for themselves that it's not what they've already read.

I read Spohn right away, and I'm at least planning to read some of the other references. But I'm not sure how typical I am in this regard.

Comment author: [deleted] 21 September 2012 07:50:59AM 1 point [-]

I do think having an index of related works is very valuable, for people wanting to do further readings, or figuring out exactly which parts of the Sequences are original.

The reader really shouldn't have to figure it out; it's a bit intellectually dishonest to impose that burden on the reader--to the author's reputational benefit.

Comment author: Randaly 22 September 2012 02:18:33AM 2 points [-]

In general, Eliezer did a fairly good job of citing things that he actually was drawing from, ie he didn't plagiarize often. Much of LukeProg's post was simply providing references to similar or independently invented ideas in academia, which were not directly relevant and would have been somewhat inappropriate to put in the posts.

Comment author: Epiphany 21 September 2012 04:07:06AM *  2 points [-]

Regardless of whether it's original, you're the one making rationality popular. Inspiring this many people to take more interest in rationality is a profoundly worthwhile accomplishment. The world needs teachers who can motivate them to think more clearly. I'm heartened to see your progress.

Comment author: Emile 15 September 2012 03:57:07PM 2 points [-]

The only real way for people to learn better is to go out and read Language in Thought and Action or Rational Choice in an Uncertain World.

Added to my list!

Do you have any more reading suggestions for people who have read the sequences? I read a few books recommended in the book recommendation open threads (or on irc), but was sometimes disappointed ("Thinking in Systems" is not very rigorous and formal, Nassim Taleb's "Fooled by Randomness" takes too much liberty interpreting various concepts, and I'm not a fan of books that start by telling me "don't worry I won't hurt your little brain with equations").

Comment author: CronoDAS 17 September 2012 04:01:02AM 0 points [-]

Great minds think alike?

Comment author: timtyler 15 September 2012 12:52:27AM 3 points [-]

It looks as though there is also earlier work on cooperation in one-shot prisoners dilemmas - e.g.:

Harrington, Joseph E. Jr. (1995) Cooperation in a one-shot Prisoners' Dilemma.

Heiner, Ronald Asher (2002) Robust Evolution Of Contingent Cooperation In Pure One-Shot Prisoners' Dilemmas

Comment author: Vulture 20 January 2014 06:17:57PM 1 point [-]

I think it would be beneficial for this list to be put on a wiki page, so that there can be more comprehensive and collaborative cross-referencing.

Comment author: theduffman 31 October 2012 10:23:37AM 0 points [-]

http://lesswrong.com/lw/va/measuring_optimization_power/ and a couple of posts before and after are variations on the ideas of Daniel Dennett's The Intentional Stance. I loved both versions.

Comment author: Bruno_Coelho 15 September 2012 06:24:39PM -1 points [-]

Even when E ideas are extremely similar to some SEP article, he assumes: " If the words are not the same, then there are differences". This sentiment converge to point 2.

I suppose has to do with possible linguistic traps. If we use terms who has a focal points as a premisse, probably these words come with hidden inferences associated to specific groups. Avoiding academic parlance whenever possible is good, but obvious improvements could be done in the writings.

Comment author: Thomas 15 September 2012 07:30:44AM 0 points [-]

What is originally his? AFAIK the FOOM and the Friendliness are his.

I am just curious.

Comment author: CarlShulman 15 September 2012 09:20:57AM 9 points [-]

Eliezer credits Nick Bostrom with coming up with the idea of Friendly AI first (and indeed while Eliezer was indifferent to AI risk on the assumption that either superintelligences would be automatically supermoral or it didn't matter what happened). FOOM probably goes to I.J. Good, or SF (Eliezer found out about the idea of a technological singularity by reading Vernor Vinge's science fiction, and closely related ideas are decades older in SF).

Comment author: knb 15 September 2012 09:16:22AM 5 points [-]

FOOM (AKA Intelligence Explosion) was formulated by I.J. Good about 50 years ago.

Comment author: Eliezer_Yudkowsky 15 September 2012 02:12:54PM 4 points [-]

...and pre-formulated by John W. Campbell, a famous science-fiction editor.

Comment author: Thomas 15 September 2012 10:12:36AM *  -1 points [-]

Maybe it is just me. But as I understood I.J. Good's intelligence explosion is much more "Kurzweilian". Happens as a consequence of some large improvement all over the place. While for the Yudkowsky's FOOM, a right binary string in the RAM of the PC from 2000 would suffice to blow us away.

I think, that the computer may need to be from today, or even from tomorrow, but this does not change much.

Comment author: timtyler 16 September 2012 04:10:12PM *  -2 points [-]

You would need improvements in both software and hardware to compete with natural nanotechnology at its best.

Improvements in software would catalyse improvements in hardware - and visa versa. I think most of the parties involved are on the same page about all this.