Wow this is awesome. Some comments and questions:
The kind of AI cooperation discussed by Eliezer is not the type discussed as "program equilibrium".
Fixed, thanks.
In your research, did you fail to find previous academic work for some elements of the sequences?
I didn't look very hard. I merely thought about the stuff I already knew about, and then picked a subset of those things to list here.
Some readers will mistakenly think that common Less Wrong views are more parochial than they really are.
This one is probably important. With non-LW newcomers to my meetup, I find explaining that we draw most of our ideas from LW kind of weird in the sense that 50% of my simulations of them conclude "these guys are some wacko internet cult". Only some of them come back.
I'd like to see at least some work on how to talk about LW without implying insularity. We can't just drop the LW affiliation, because nowhere else really compares (even if everything on LW exists somewhere else, it doesn't exist anywhere else all in the same place).
HI WE'RE AN INTERNET MEETUP GROUP!
The internet is low status due to the low barriers to entry. Mention higher status things than the internet.
Future of Humanity Institute at Oxford (Hey I've heard of Oxford)
Vinge (a published author many have heard of)
Center for Applied Rationality which does real things in real life with real people
Talk about cognitive science. Talk about economics. Talk about anything but the internet. LessWrong? Oh it's just for coordinating all the interesting people who are interested in these interesting things.
I'd like to see at least some work on how to talk about LW without implying insularity.
Name-drop like a motha...
"Did you read Predictably Irrational by Dan Ariely or Thinking Fast and Slow by Nobel laureate Daniel Kahneman? We study their field of predictable human thinking errors and try to figure out how best to apply those lessons to everyday human life so that we can learn how to make decisions that are more likely to achieve our goals.
"We talk some about Alan Turing's idea that machines could one day become smarter than humans, and how shortly thereafter we might expect them to become more powerful than humans. One of the mathematicians who worked with him to crack the German Enigma Code, I.J. Good, explained that a smarter-than-human machine could use its intelligence to improve its own inteligence. And since neuroscientists like Paul Glimcher at NYU and Kent Berridge at U Michigan are learning that what humans care about is incredibly complex, it's unlikely that we'll be able to figure out how to program smarter-than-human machines to respect every little detail of what we care about."
Some readers will mistakenly think that common Less Wrong views are more parochial than they really are.
I think the parochialism comes from high handed smack-talk like "The obvious answer to philosophically recondite issue is X, and all you need to see this is obvious is our superior rationality". Best example here.
One of the easiest hard questions, as millennia-old philosophical dilemmas go. Though this impossible question is fully and completely dissolved on Less Wrong, aspiring reductionists should try to solve it on their own.
I get a similar vibe regarding QM (obviously many worlds), religion (obviously atheism), phil of mind (obviously reductionsim), and (most worrying) ethics and meta-ethics.
The fact the candidate views espoused are part of the academic mainstream doesn't defray the charge of parochialism due to the tup-thumping, uncharitable-to-opponents and generally under-argued way these views are asserted. Worse, it signals lack of competence on the part of LW: given the views of virtually all domain experts on any of these things, your degree of confidence is better explained by inferior, not superior knowledge, and even if you happen to get the right answer, I doubt you're p-reliable or tracking.
I don't think there's much value in pretending that issues like God (and the absence thereof) or the compatibility between determinism and (any logically coherent view of) free will haven't been decisively answered.
Seriously now, the compatibility between free will and determinism is something that I was figuring out by myself back in junior high. Eliezer with his "Thou Art Physics" expressed it better and more compactly than I ever did to myself (I was instead using imagery of the style "we're the stories that write themselves", and this was largely inspired by Tolkien's Ainulindale, where the various gods sing a creation song that predicts all their future behaviour), but the gist is really obvious once you get rid of the assumption that determinism and free will must somehow be opposed.
In every discussion I've had since, in any forum, nobody who thinks them to be incompatible can describe even vaguely what "free will" would be supposed to look like if it does not contain determinism inside it.
I think this is a case of exactly the problem I diagnosed above.
Compatibilism (and related views) have been mentioned at least since Hume, and have been discussed extensively in modern analytic philosophy. Although it commands a slender majority of philosophers of action, it is not like the entire philosophical community considers compatibilism obviously or decisively the 'right answer' (see here, and here for a long index of reasons/objections etc.). You'd be pretty hard pressed to find a single philosopher of action who considers free will a 'solved problem'.
Yet it seems the less wrong community considers it solved based on a sequence of blog posts which merely explicates compatibilism: I couldn't find any discussion of compatibilism which goes beyond undergrad philosophy level, no discussion of common objections to compatibilism, engagement with any thinkers arguing against, nothing.
The two best explanations I have for this is either compatibilism is just obvious and people of sufficient rationality can be confident that domain experts on free will who don't buy compatibilism are wrong, or that the LW 'solution' is frankly philosophically primitive but LWers are generally too f...
A weak majority/strong plurality of relevant domain experts does not make the question decisively answered. I don't have survey data on this, but I'm pretty sure none of the compatibilists (even those who 'accept' it), take the question to be obviously answered etc. etc.
But more importantly: these people also seem to prefer to two-box in Newcomb's problem. So why should I put much weight in their opinion?
The majority of decision theory specialists two-box. I'm sure you can guess what I'm going to say about doman expertise and dunning-kruger effect here, too.
Tell me, do you have any criterion over whether something is "decisively answered" other than how many "relevant domain experts" agree with it? If your definition of "decisively answered" is solely dependent on this, then we can just agree that we were using different definitions for the term.
I take 'decisively answered' to mean something along the lines of "here is an account, which, properly understood, solves this problem to the satisfaction of reasonable people". So (near) unanimity among relevant domain experts is necessary but not sufficient for this. I can't think of anything in natural language we would call a 'decisive answer' or similar in which 40% or so of relevant domain experts disagree with.
So much for the decision theory specialists. Implement a real life version of Newcomb's box, where you fill in the opaque box based on whether they said they'll one-box or two-box. Assuming everyone follows what they said they should do, the one-boxers will just win, and the two-boxers will be weeping.
This is recapitulating a standard argument for one-boxing, and it is well discussed in the literature. The fact the bulk of people who spend their time studying this issue and don't find this consideration decisive should make you think it is less a silver bullet than you think it is.
What about Drescher's Good and Real: Demystifying Paradoxes from Physics to Ethics? Eliezer said it's "pratically Less Wrong in book form."
I'd also mention
Eliezer has explicitly mentioned Hayakawa in Intensions and Extensions. Axelrod is important to understanding the iterated Prisoner's Dilemma. Singer is one of the better-known proponents of a "shut up and multiply" approach to utilitarianism, agreeing with Eliezer's conclusions in various places (eg). Rawls' notions of the "veil of ignorance" and "reflective equilibrium" have been mentioned in connection with Coherent Extrapolated Volition - when I first came across CEV the similarities with Rawls stuck out like a sore thumb.
And remember, billionaire friend Peter Thiel financed this immense production, which E.Y. wrote while on Thiel's payroll at SIAI. It's immensity isn't a product of original thought but, in the end, of a pampered parasite with too much time on his hands.
Seriously? DevilWorm is criticizing Eliezer for managing to find financial support while producing his intellectual output? That's how most intellectual work gets done---both within and without formal academic establishments and in various forms across time. This criticism is utterly pathetic.
I don't understand how DevilWorm has been upvoted here (+3 prior to my vote). The remainder of the comment is almost as bad---as is every other comment he has written. Moreover a brief glance at the user's comment history identifies the DevilWorm account as rather obviously yet another sock-puppet created for the purpose of acting out his personal animosity.
How the heck are our new anti-trolling measures going to work if people go around upvoting blatant trolling like this? I mean sure, this guy seems to have a basic grasp of grammar and punctuation but apart from that he doesn't seem to have contributed anything but some filler in which to embed some unnecessary insults.
DevilWorm's comments (and all other slander-sockpuppets) should not be upvoted. They should be downvoted to invisibility until someone has a chance to ban the comments---and that only because the 'ban user by name and IP' is missing.
And lo, people began tweeting:
Which is false. This pushes as far in the opposite wrong direction as the viewpoint it means to criticize.
Evolutionary biology, the non-epistemological part of the exposition of quantum mechanics, and of course heuristics and biases, are all not original. They don't look deceptively original either; they cite or attributed-quote the sources from which they're taken. I have yet to encounter anyone who thinks the Sequences are more original than they are.
When it comes to the part that isn't reporting on standard science, the parts that are mostly dealt with by modern "philosophers" rather than experimental scientists of one kind or another, the OP is vastly overstating how much of the Sequences are similar to the standard stuff out there. There is such a vast variety of philosophy that you can often find a conclusion similar to anything, to around the same degree that Leibniz's monadology anticipated timeless quantum mechanics, i.e., not very much. The motivations, the arguments by which things are pinned down, the exact form of the conclusions, and what is done with thos...
the OP is vastly overstating how much of the Sequences are similar to the standard stuff out there... I think Luke is being extremely charitable in his construal of what's "already" been done in academia
Do you have a Greasemonkey script that rips all the qualifying words out of my post, or something? I said things like:
Your comment above seems to be reacting to a differ...
Most people won't read my references. But some of those who do will go on to make a sizable difference as a result. And that is one of the reasons I cite so many related works, even if they're not perfectly identical to the thing me or somebody else is doing.
FWIW, Luke's rigorous citation of references has been absurdly useful to me when doing my research. It's one of the aspects of reading LW that makes it worthwhile and productive.
Luke is already aware that I've utilized his citations to a great extent, but I wanted to publicly thank him for all that awesome work. I'd also like to thank others who have done similar things, such as Klevador. We need more of this.
I am trying to counteract these three effects
I think a valid criticism can be made that while you were trying to counteract these three effects (which is clearly an important and useful effort), you didn't take enough care to avoid introducing a new effect, of making some people think the Sequences are less original than they actually are. (For example you didn't ask Eliezer to double check your descriptions of how the Sequences posts relate to the academic works, and you didn't give some examples of where the Sequences are original.)
This is bad because in addition to communicating various ideas, the Sequences also serve as evidence of Eliezer's philosophy and rationality talents/skills, which is useful for potential donors/supporters to judge the likely future effectiveness of the Singularity Institute in achieving its goals.
All readers have a Greasemonkey script that rips all the qualifying words out of a post... I don't see why on Earth you'd expect any other message to come across than ["Eliezer's Sequences are mostly not original"].
Do you think most readers misinterpreted my post in that way? I doubt it. It looks to me like one person tweeted "Eliezer's Sequences mostly not original" — a misinterpretation of my post which I've now explicitly denied near the top of the post.
My guess now would be that I probably underestimate the degree to which readers misinterpreted my post (because my own intentions were clear in my mind, illusion of transparency), and that you probably overestimate the degree to which readers misinterpreted my post (because you seem to have initially misinterpreted it, and that misinterpretation diminishes several years of cognitive work that you are justly proud of).
Also: you seem to be focusing on the one tweeted misinterpretation and not taking into account that we have evidence that the post is also achieving its explicitly stated goals, as evidenced by many of the comments on this thread: 1, 2, 3, 4, 5.
It is very easy to read the sequences and think that you think the philosophical thought is original to you. Other than the FAI stuff and decision theory stuff, is that true?
What exactly is wrong with being thought of as a very high-end popularizer? That material is incredibly well presented.
Additionally, people who disagree with your philosophical positions ought not be put in the (EDIT: position) of needing to reinvent the philosophical wheel to engage critically with your essays.
Only a single conclusion is possible: LukeProg is a TRAITOR!
I can understand why this would be negatively received by some---it is clearly hyperbole with a degree of silliness involved. That said---and possibly coincidentally---there is a serious point here. In fact it is the most salient point I noticed when reading the post and initial responses.
In most social hierarchies this post would be seen as a betrayal. An unusually overt and public political move against Eliezer. Not necessarily treason, betrayal of the tribe, it is a move against a rival. Of course it would certainly be in the interest of the targeted rival to try to portray the move as treason (or heresy, or whatever other kind of betrayal of the tribe rather than mere personal conflict.)
The above consideration is why I initially expected Eliezer to agree to a larger extent than he did (which evidently wasn't very much!) Before making public statements of a highly status sensitive nature regarding an ally the typical political actor will make sure they aren't offending them---they don't take the small risk establishing an active rivalry unless they are certain the payoffs are worth it.
This (definitely!) isn't to say ...
The thought that Luke was trying to sabotage my position, consciously or unconsciously, honestly never crossed my mind until I read this comment. Having now considered the hypothesis rather briefly, I assign it a rather low probability. Luke's not like that.
With both your comment here and your comments on the troll-fee issue I've found you coming across as arrogant. This perception seems to roughly match the response that other people have had to those comments as well, since most people disagreed with you in both areas (judging by number of upvotes). I hadn't perceived you that way before now, so I'm wondering if something happened to you recently that's altered the way you post or the way you think. This change is for the worse; I want my old model of Eliezer Yudkowsky back!
Frankly, I have found the sequences to be primarily useful for condensing concepts that I already had inside my head. The ideas expressed in almost all of the sequences are blatantly obvious, but they come across as catchy and often are reducible to a quick phrase. Their value lies in the fact that they make it easy to internalize certain ideas so that they're more readily accessible to me. They also helped clarify the boundaries of some concepts, to a certain extent. The sequences have provided me with a useful terminology, but I don't think they've offered me much else.
What ideas do you believe to be original that you've produced?
Is there a reason that defending the originality of the sequences is so important to you?
With both your comment here and your comments on the troll-fee issue I've found you coming across as arrogant.
You only got this now?
One anecdote given the 'PR' worries raised:
I have never read the sequences. After reading Luke's post, I am much less likely to: the impression given is the sequences are generally idiosyncratic takes which recapitulate an already existing and better organized literature. I also think it is more likely the sequences are overrated, either through readers being unaware their (or similar) insights have already been made, or lacking the technical background to critique them.
It also downgraded my estimate of the value of EY's work. Although I was pretty sceptical, I knew there was at least some chance that the sequences really were bursting with new insights and that LW really was streets ahead of mainstream academia. This now seems much less likely - although I don't think EY is a plagiarist, it seems most of the sequences aren't breaking new ground, but summarizing/unwittingly recapitulating insights that have already been made and taken further elsewhere.
So I can see the motivation for EY to defend that their originality: his stock goes down if the sequences are neat summaries but nothing that new rather than bursting with new and important insights, and EY's stock is important for things like donations, public perception of him and the SI, etc. (Both my likelihood of donating and my regard for SI has been lowered a bit by this post and comments). However, EY's way of responding to (weakly implied) criticism with catty arrogance compounds the harm.
If you are at all interested in rationality it would be a huge shame for you to skip the Sequences.
Yes, a lot of the material in the Sequences could also be obtained by reading very very carefully a few hundred impenetrable scholarly books that most people have never heard of in five or ten different disciplines, supplemented by a few journal articles, plus some additional insights by "reading between the lines", plus drawing all the necessary connections between them. But you will not do this.
The Sequences condense all that information, put it in a really fun, really fascinating format, and transfer all of it into the deepest levels of your brain in a way that those hundred books wouldn't. And then there's some really valuable new material. Luke and Eliezer can argue whether the new material is 30% of the Sequences or 60% of the Sequences, but either number is still way more output than most people will produce over their entire lives.
If your worry is that they will just be recapitulating things you already know, I am pretty doubtful; I don't know your exact knowledge level, but they were pretty exciting for me when I first read them and I had college degrees in philosop...
The sequences need a second edition. It's sheer hubris to think that nothing has changed in four years.
There would be room for improvement even without anything changing. They were produced as daily blog posts for the purpose of forcing Eliezer to get his thoughts down on a page.
I hadn't expected you to disagree with that tweet, so I'm clearly getting something wrong. I wrote that in the hope that it would encourage people to read the Sequences, not put them off - I think people imagine it as this million-word work of revelation, but a very large part of what it is is a work of popular science - turning people on to good existing ideas in psychology and philosophy and biology and physics and suchlike. There is a great deal that is original and valuable in there, but I don't think of it as the majority of the material.
I get your point, but to lots of people the wording of that tweet would have the connotation ‘EY is a plagiarist’, not ‘EY is not a crackpot’.
The word "original" has positive connotations. And therefore the words "unoriginal" or "not original" have negative connotations.
So, yeah, I don't think you'd encourage anyone to read anything by calling it "not original".
Thinking about it further though, this makes something of a nonsense of the original tweet, since it's hard to think what would count as "mostly original" by this standard. You might as well describe eg The Better Angels of Our Nature as "mostly not original" since it contains no original research but presents a synthesis of the research of others, building up to a common theme.
The problem I have is that if I say something that sounds positive about the Sequences, that's going to turn my friends off, since they already know I think well of them. By saying something that on first reading sounds negative, I might get their interest, but that only works if they go on to follow the link.
The problem I have is that if I say something that sounds positive about the Sequences, that's going to turn my friends off, since they already know I think well of them.
For example, they may be turned off if you came out and said "The sequences really aren't the parochial ramblings of an intellectual outcast, they are totally in accord with mainstream scientific thinking". But "mostly not original" conveys much of the same message by making a concession to the orthodoxy.
I agree that Luke's post might cause some people to update too much in the direction of "the Sequences aren't original". He was wrong or overstated things in the couple of bullet points that I checked out (and pointed out in my earlier comment). He probably should have showed it to you for error-checking and making sure it's being fair before posting it.
I do think having an index of related works is very valuable, for people wanting to do further readings, or figuring out exactly which parts of the Sequences are original.
So they won't actually read the literature and find out for themselves that it's not what they've already read.
I read Spohn right away, and I'm at least planning to read some of the other references. But I'm not sure how typical I am in this regard.
There's literature out there which is written in the same spirit as LW, but with different content. Now that's an exciting message. It might even get people to read things.
Maybe we can start to build up a repository of those things, too. So far, you've recommended:
Unfortunately, those works seem incredibly different to me, so it's hard for me to guess which other works you would also endorse as being in the "LW spirit." I'll try anyway:
There's a spirit in LW which really is a spirit that exists in many other places,
Yes, and pointing out those other places here serves two purposes.
It serves to brand LW, so that people passing by can quickly see the kind of spirit here. Yes, there's a whole world out there, and many of us have spent some time in it, so seeing references to that world here serves to quickly communicate some of what LW is about.
References also server to point people here to other expositions of similar material.
For example, you say:
The only real way for people to learn better is to go out and read Language in Thought and Action
I'd recommend people at some point move on from Hayakawa to Korzybski, Science and Sanity, and the whole General Semantics literature. People have spent decades discussing these issues and organizing
It doesn't get people to read the literature. Why should they? From what they can see, it's already been presented to them on LW, after all.
That's not my reaction to references. When I first came here, the references to Jaynes didn't make me think "I've already covered this stuff, no need to read this web site." On the contrary, it made me want to read more....
TDT doesn't artifically sever decision nodes from anything upstream; the idea is that observing your algorithm, but not its output, is supposed to screen off things upstream.
Pardon me; I'm not yet much of an expert with LW decision theories. When you explained TDT on the whiteboard to Alex (with me listening), you kept talking about "severing" rather than "screening off." I'll try to find a way to modify the OP.
ETA: I remembered I have a recording of that tutorial, and I when checked the recording, and it turns out my memory was wrong. You did talk about how TDT "screened off" the information whereas CDT "severs" the causal diagram.
I don't understand what the purpose of this post was supposed to be - what positive consequence it was supposed to have.
I took the post to be Luke writing notes to himself, in public so as to recruit others' help, toward the kind of bibliography that might be included in an academically acceptable version of the Sequences, or of some parts of them.
The intention being, I gathered, to publish these bibliographies as an adjunct to the Sequences - perhaps in the "wall of references" style of Luke's early posts. (If so, I hope a more user-friendly way of displaying those is worked out first!)
(ETA: the specific positive consequence of that would be to help the reader "find the related works in academia" as per Luke's third numbered point in the OP.)
I'm stating what I discerned of the intention - I won't presume to judge the OP either as a plan of action, or as a first step in its execution.
Completely agree with your latter addendum that people should read Hofstadter, Hayakawa etc. not as footnotes to your work but for their own merits. Hofstadter I discovered in childhood and I wouldn't be the same person if I hadn't; I read Hayakawa on your recommendation, and am glad I did. Yay to more discussion of works that have the LW-nature, but are not otherwise alluded to in the Sequences. :)
Thankyou for clearing that up. Given your occupational affiliation with Luke I had been overestimating the extent to which you would endorse his position. That is, I wouldn't have expected Luke to write this without checking with you first so thought you must have agreed.
It's posts like this that remind me that the sequences are vast, excellent, and most importantly of all, not particularly organized at the moment.
Every so often, Lukeprog or others will make a small effort towards collating the sequences, but the resulting product disappears into the ether of Discussion archives.
Talk is cheap, but somebody really needs to do something about the sequences to make them more accessible and visible to a newcomer. The LW wiki index of the sequence is incomplete, and seems like it hasn't been changed since 'Tetronian' created it six months ago.
Likewise, much of the Quantum Physics sequence can be found in quantum physics textbooks, e.g. Sakurai & Napolitano (2010).
I don't think Sakurai is the best reference here - most of an introductory QM book will be about what particles do in the presence of forces, and treats identical particles in a more complicated language because they can be either fermions or bosons.
A better text would be an introduction for people who want to do quantum computing - those people get to use all the nice abstractions and let the physicists worry about the particle...
I think it would be beneficial for this list to be put on a wiki page, so that there can be more comprehensive and collaborative cross-referencing.
This is good. Getting people to read lots of quality stuff by a wide variety of authors can put them on the path of being able to produce high quality output themselves, after overlearning the concepts and ways to present them from many viewpoints.
People who just stick with the sequences can end up parroting the surface jargon and alienating people who expect familiarity with a bit wider range of literature for someone whose opinion they would value.
So, you approve of the practice of disappearing comments without any notice of the fact or the reason?
Not as such, but I approve of disappearing anything everything from known trolling sockpuppet accounts.
(I feel like I should be paying a 5 karma troll-feeding-toll to write this but for some reason there are upvotes where I expected downvotes. I'll wait a day to see how things stabilize then consider if my model of lesswrong users needs to be updated.)
And why was it deleted, when I've posted far more "objectionable" matter?
Because the moderators don't have access to a "ban account' feature for accounts that only post 'objectionable' material.
If you were an honest person (not one feeding at the trough of Thiel/Yudkowsky), you would vote yourself down.
Assuming we disregard ownership of multiple accounts, the consequent of your conditional is impossible.
It looks as though there is also earlier work on cooperation in one-shot prisoners dilemmas - e.g.:
Harrington, Joseph E. Jr. (1995) Cooperation in a one-shot Prisoners' Dilemma.
Heiner, Ronald Asher (2002) Robust Evolution Of Contingent Cooperation In Pure One-Shot Prisoners' Dilemmas
The sheer magnitude is what impresses gullible readers of the Sequences.
Wow. Deja vu. I actually have to follow this link and double check the date to see if this was the same comment we dealt with before or just a repetition of the same agenda by the same sockpuppet. If you check DevilWorm's user page you will see that this comment is a copy and paste clone of one he previously made that has now been deleted or banned (5 comments below on that page, to be precise). Once again it has received initial upvotes---either from his other accounts or from user...
Eliezer credits Nick Bostrom with coming up with the idea of Friendly AI first (and indeed while Eliezer was indifferent to AI risk on the assumption that either superintelligences would be automatically supermoral or it didn't matter what happened). FOOM probably goes to I.J. Good, or SF (Eliezer found out about the idea of a technological singularity by reading Vernor Vinge's science fiction, and closely related ideas are decades older in SF).
http://lesswrong.com/lw/va/measuring_optimization_power/ and a couple of posts before and after are variations on the ideas of Daniel Dennett's The Intentional Stance. I loved both versions.
That's why he always refuses to summarize his conclusions.
It seems like his latest sequence is offering summarized versions of at least some of the previous sequences.
Looking at the PhilPapers.org survey of philosophers, the pattern of views in the main sequences can be predicted rather well with one heuristic: select the option that (religious) philosophers of religion disproportionately reject. Or, alternatively, the option most compatible with Dennett-style naturalism. Exceptions include one-boxing on Newcomb, and perhaps personal identity (which is somewhat tied up with transhumanism, but judged differently by philosophers of computing vs biology).
However, the conjunction of all of these naturalist-favored views has...
limp on for years
More like a few weeks.
All I can do now is be silent. I shall sink into the divine shadow, and will not know the troll or the metatroll, or anything else. Numquam novi nomen eius, nomen troglodytam.
Even when E ideas are extremely similar to some SEP article, he assumes: " If the words are not the same, then there are differences". This sentiment converge to point 2.
I suppose has to do with possible linguistic traps. If we use terms who has a focal points as a premisse, probably these words come with hidden inferences associated to specific groups. Avoiding academic parlance whenever possible is good, but obvious improvements could be done in the writings.
Due in part to Eliezer's writing style (e.g. not many citations), and in part to Eliezer's scholarship preferences (e.g. his preference to figure out much of philosophy on his own), Eliezer's Sequences don't accurately reflect the close agreement between the content of The Sequences and work previously done in mainstream academia.
I predict several effects from this:
I'd like to counteract these effects by connecting the Sequences to the professional literature. (Note: I sort of doubt it would have been a good idea for Eliezer to spend his time tracking down more references and so on, but I realized a few weeks ago that it wouldn't take me much effort to list some of those references.)
I don't mean to minimize the awesomeness of the Sequences. There is much original content in them (edit: probably most of their content is original), they are engagingly written, and they often have a more transformative effect on readers than the corresponding academic literature.
I'll break my list of references into sections based on how likely I think it is that a reader will have missed the agreement between Eliezer's articles and mainstream academic work.
(This is only a preliminary list of connections.)
Obviously connected to mainstream academic work
Eliezer's posts on evolution mostly cover material you can find in any good evolutionary biology textbook, e.g. Freeman & Herron (2007).
Likewise, much of the Quantum Physics sequence can be found in quantum physics textbooks, e.g. Sakurai & Napolitano (2010).
An Intuitive Explanation of Bayes' Theorem, How Much Evidence Does it Take, Probability is in the Mind, Absence of Evidence Is Evidence of Absence, Conservation of Expected Evidence, Trust in Bayes: see any textbook on Bayesian probability theory, e.g. Jaynes (2003) or Friedman & Koller (2009).
What's a Bias, again?, Hindsight Bias, Correspondence Bias; Positive Bias: Look into the Dark, Doublethink: Choosing to be Biased, Rationalization, Motivated Stopping and Motivated Continuation, We Change Our Minds Less Often Than We Think, Knowing About Biases Can Hurt People, Asch's Conformity Experiment, The Affect Heuristic, The Halo Effect, Anchoring and Adjustment, Priming and Contamination, Do We Believe Everything We're Told, Scope Insensitivity: see standard works in the heuristics & biases tradition, e.g. Kahneman et al. (1982), Gilovich et al. 2002, Kahneman 2011.
According to Eliezer, The Simple Truth is Tarskian and Making Beliefs Pay Rent is Peircian.
The notion of Belief in Belief comes from Dennett (2007).
Fake Causality and Timeless Causality report on work summarized in Pearl (2000).
Fake Selfishness argues that humans aren't purely selfish, a point argued more forcefully in Batson (2011).
Less obviously connected to mainstream academic work
Eliezer's metaethics sequences includes dozens of lemmas previously discussed by philosophers (see Miller 2003 for an overview), and the resulting metaethical theory shares much in common with the metaethical theories of Jackson (1998) and Railton (2003), and must face some of the same critiques as those theories do (e.g. Sobel 1994).
Eliezer's free will mini-sequence includes coverage of topics not usually mentioned when philosophers discuss free will (e.g. Judea Pearl's work on causality), but the conclusion is standard compatibilism.
How an Algorithm Feels From Inside and Dissolving the Question suggest that many philosophical problems can be dissolved into inquiries into the cognitive mechanisms that produce them, as also discussed in, for example, Shafir (1998) and Talbot (2009).
Thou Art Godshatter, Not for the Sake of Happiness Alone, and Fake Utility Functions make the point that value is complex, a topic explored in more detail in affective neuroscience (Kringelbach & Berridge 2009), neuroeconomics (Glimcher 2010; Dolan & Sharot 2011), and other fields.
Newcomb's Problem and the Regret of Rationality repeats a common debate among philosophers. Thinking that CDT must be right even though it "loses" to EDT on Newcomb's Problem, one group says "What can we do, if irrationality is rewarded?" The other group says "If you're so smart, why aren't you rich? What kind of rationality complains about the reward for irrationality?" For example, see Lewis (1981).
I don't think Eliezer had encountered this mainstream work when he wrote his articles
Eliezer's TDT decision algorithm (2009, 2010) had been previously discovered as a variant of CDT by Wolfgang Spohn (2003, 2005, 2012). Both TDT and Spohn-CDT (a) use Pearl's causal graphs to describe Newcomblike problems, then add nodes to those graphs to represent the deterministic decision process the agent goes through (Spohn calls them "intention nodes," Yudkowsky calls them "logical nodes"), (b) represent interventions at these nodes by severing (edit: or screening off) the causal connections upstream, and (c) propose to maximize expected utility by summing over possible values of the decision node (or "intention node" / "logical node"). (Beyond this, of course, there are major differences in the motivations behind and further development of Spohn-CDT and TDT.)
Many of Eliezer's points about intelligence explosion and machine ethics had been made in earlier writings Eliezer did cite, e.g. Williamson (1947), Good (1965), and Vinge (1993). Others of Eliezer's points appear in earlier writings he did not cite but probably had read: e.g. Minsky (1984), Schmidhuber (1987), Bostrom (1997), Moravec (1999). Others of Eliezer's points appear in earlier writings he probably hadn't read: e.g. Cade (1966), Good (1970), Versenyi (1974), Lukasiewicz (1974), Lampson (1979), Clarke (1993, 1994), Sobel (1999), Allen et al. (2000). (For a brief history of these ideas, see here and here.)
A Technical Explanation of Technical Explanation retreads much ground from the field of Bayesian epistemology, surveyed for example in Niiniluoto (2004) and Howson & Urbach (2005).