Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Rationality: From AI to Zombies

81 Post author: RobbBB 13 March 2015 03:11PM

 

Eliezer Yudkowsky's original Sequences have been edited, reordered, and converted into an ebook!

Rationality: From AI to Zombies is now available in PDF, EPUB, and MOBI versions on intelligence.org (link). You can choose your own price to pay for it (minimum $0.00), or buy it for $4.99 from Amazon (link). The contents are:

  • 333 essays from Eliezer's 2006-2009 writings on Overcoming Bias and Less Wrong, including 58 posts that were not originally included in a named sequence.
  • 5 supplemental essays from yudkowsky.net, written between 2003 and 2008.
  • 6 new introductions by me, spaced throughout the book, plus a short preface by Eliezer.

The ebook's release has been timed to coincide with the end of Eliezer's other well-known introduction to rationality, Harry Potter and the Methods of Rationality. The two share many similar themes, and although Rationality: From AI to Zombies is (mostly) nonfiction, it is decidedly unconventional nonfiction, freely drifting in style from cryptic allegory to personal vignette to impassioned manifesto.

The 333 posts have been reorganized into twenty-six sequences, lettered A through Z. In order, these are titled:

  • A — Predictably Wrong
  • B — Fake Beliefs
  • C — Noticing Confusion
  • D — Mysterious Answers
  • E — Overly Convenient Excuses
  • F — Politics and Rationality
  • G — Against Rationalization
  • H — Against Doublethink
  • I — Seeing with Fresh Eyes
  • J — Death Spirals
  • K — Letting Go
  • L — The Simple Math of Evolution
  • M — Fragile Purposes
  • N — A Human's Guide to Words
  • O — Lawful Truth
  • P — Reductionism 101
  • Q — Joy in the Merely Real
  • R — Physicalism 201
  • S — Quantum Physics and Many Worlds
  • T — Science and Rationality
  • U — Fake Preferences
  • V — Value Theory
  • W — Quantified Humanism
  • X — Yudkowsky's Coming of Age
  • Y — Challenging the Difficult
  • Z — The Craft and the Community

Several sequences and posts have been renamed, so you'll need to consult the ebook's table of contents to spot all the correspondences. Four of these sequences (marked in bold) are almost completely new. They were written at the same time as Eliezer's other Overcoming Bias posts, but were never ordered or grouped together. Some of the others (A, C, L, S, V, Y, Z) have been substantially expanded, shrunk, or rearranged, but are still based largely on old content from the Sequences.

One of the most common complaints about the old Sequences was that there was no canonical default order, especially for people who didn't want to read the entire blog archive chronologically. Despite being called "sequences," their structure looked more like a complicated, looping web than like a line. With Rationality: From AI to Zombies, it will still be possible to hop back and forth between different parts of the book, but this will no longer be required for basic comprehension. The contents have been reviewed for consistency and in-context continuity, so that they can genuinely be read in sequence. You can simply read the book as a book.

I have also created a community-edited Glossary for Rationality: From AI to Zombies. You're invited to improve on the definitions and explanations there, and add new ones if you think of any while reading. When we release print versions of the ebook (as a six-volume set), a future version of the Glossary will probably be included.

Comments (96)

Comment author: Persol 01 April 2015 01:17:22AM *  28 points [-]

Perhaps this is already discussed elsewhere and I'm failing at search. I'd be amazed if the below wasn't already pointed out.

On rereading this material it strikes me that this text is effectively inaccessible to large portions of the population. When I binged on these posts several years ago, I was just focused on the content for myself. This time, I had the thought to purchase for some others who would benefit from this material. I realized relatively quickly that the purchase of this book would likely fail to accomplish anything for these people, and may make a future attempt more difficult.

I think many of my specific concerns apply to a large percentage of the population.

  • The preface and introductions appear aimed at return readers. The preface is largely a description of 'oops', which means little to a new reader and is likely to trigger a negative halo effect in people who don't yet know what that means. - "I don't know what he's talking about, and he seems to make lots of writing mistakes."
  • There isn't a 'hook'. Talking about balls in urns in the intro seems too abstract for people. The rest of the sequences have more accessible examples, which most people would never reach.
  • Much of the original rhetoric is still in place. Admittedly that's part of what I liked about the original posts, but I think it limits the audience. As a specific example, a family member is starting high school, likes science, and I think would benefit from this material. However her immediate family is very religious, to the point of 'disowning' a sister when they found out about an abortion ~25 years ago. The existing material uses religion as an example of 'this is bad' frequently enough that my family member would likely be physically isolated from the material and socially isolated from myself. 87% of America (86% global) have some level of belief in religion. The current examples are likely to trigger defensive mechanisms, before they're education about them. (Side-note: 'Waking Up: A Guide to Spirituality Without Religion – by Sam Harris' is a good book, but has this same exact issue.)
  • Terminology is not sufficiently explained for people seeing this material with fresh eyes. As an example, ~15% of the way through 'New Improved Lottery' talks about probability distributions. There was no previous mention of this. Words with specific meanings, that are now often used, are unexplained. 'Quantitative' is used and means something to us, but not to most people. The Kindle provided dictionary and Wikipedia definitions are not very useful. This applies to the chapter titles as well, such as 'Bayesian Judo'.
  • The level of hyperlinks, while useful for us, is not optimal for someone reading a subject for the first time. A new reader would have to switch topics in many cases to understand the reference.
  • References to LessWrong and Overcoming Bias and only make sense to us.

Eliezer and Robb have done a lot to get the material into book state... but it's preaching to the choir.

Specifically what I think would make this more accessible:

  • A more immediate hook along the lines of 'Practicing rationality will help you make more winning decisions and be less wrong.' (IE: keep reading because this=good and doable) Eliezer was prolific enough that I think good paragraphs likely already exist; but need connectors.
  • Where negative examples are likely to dissuade large numbers of people, find better examples. In general avoid mentions of specific politics or religion in general. It's better to boil the frog.
  • Move or remove all early references to Bayes. 'Beliefs that are rational are call Bayesian' means nothing to most people. Later references might as well be technobabble.
  • Make sure other terminology is actually explained/understandable before it's used in the middle of an otherwise straightforward chapter. I'd try 1n & 2n-gramming the contents against Google Ngrams to identify terminology we need to make sure is actually explained/understood before casual use.
  • Get this closer to a 7th grade reading level. This sets a low bar at potential readers who can understand 'blockbuster' books in English. (This might be accomplished purely with the terminology concern/change above)
  • Change all hyperlinks to footnotes.
  • Discuss LessWrong, Overcoming Bias, Eliezer, Hanson in the preface as 'these cool places/people where much of this comes from' but limit the references within the content.

Is there any ongoing attempt or desire to do a group edit of this into an 'Accessible Rationality'?

Comment author: RobbBB 01 April 2015 10:30:19PM *  13 points [-]

Thanks for all the comments! This is helpful. I agree 'Biases: An Introduction' needs to function better as a hook. The balls-in-an-urn example was chosen because it's an example Eliezer re-uses a few times later in the Sequences, but I'd love to hear ideas for better examples, or in general a more interesting way to start the book.

'Religion is an obvious example of a false set of doctrines' is so thoroughly baked into the Sequences that I think getting rid of it would require creating an entirely new book. R:AZ won't be as effective for theists, just as it won't be as effective for people who find math, philosophy, or science aversive.

I agree with you about 'boiling the frog', though: it would be nice if the book eased its way into anti-religious examples. I ended up deciding it was more important to quickly reach accessible interesting examples (like the ones in 'Fake Beliefs') than to optimize for broad appeal to theists and agnostics. One idea I've been tossing around, though, is to edit Book I ('Map and Territory') and Book II ('How to Actually Change Your Mind') for future release in such a way that it's possible to read II before I. It will still probably be better for most people to start with I, but if this works perhaps some agnostic or culturally religious readers will be able to start with II and get through more content before running into a huge number of anti-religious sentiments.

I agree about doing more to address the technobabble. In addition to including a Glossary in future editions of the book, I'll look into turning some unnecessarily technical asides into footnotes. The hyperlinks, of course, will need to be removed regardless when the print book comes out.

Comment author: Gram_Stone 01 April 2015 05:24:47AM 9 points [-]

I've had similar concerns and I agree with a lot of this.

Get this closer to a 7th grade reading level. This sets a low bar at potential readers who can understand 'blockbuster' books in English. (This might be accomplished purely with the terminology concern/change above)

If we really want to approach a 7th grade reading level, then we had better aim for kindergartners. I remember reading through the book trying to imagine how to bring it down several levels and thinking about just how many words I was taking for granted as a high-IQ adult who has had plenty of time to just passively soak up vocabulary and overviews of highly complex fields. I just don't think we're there yet; I think that's why there are things like SPARC where we're trying it out on highly intelligent high school students who are unusually well-educated for their age.

Change all hyperlinks to footnotes.

To my knowledge this is already a priority.

Is there any ongoing attempt or desire to do a group edit of this into an 'Accessible Rationality'?

I find that there's a wide disparity between LW users in intelligence and education, and I don't know if I see a wiki-like approach converging on anything particularly useful. I would imagine arguments about what's not simple enough and what's not complex enough, and about people using examples from their pet fields that others don't understand. It might work if you threw enough bodies at it, like Wikipedia, but we don't have that many bodies. I don't know how others feel.

Comment author: [deleted] 01 April 2015 06:57:15PM 4 points [-]

The point wasn't to aim for 7th graders, but a 7th grade level which would make it generally accessible to busy adults.

Comment author: Persol 01 April 2015 09:40:22PM 2 points [-]

See Mark's post regarding 7th grade; my intention was aimed at adults, who (for whatever reason) seem to like the 7th grade reading level.

I'm not sure how to effectively crowd source this without getting volunteers for specific (non-overlapping) tasks and sections. I share your concern with the wiki-method, unless each section has a lead. At work we regularly get 20 people to collaborate on ~100 page proposals, but the same incentives aren't available in this case. Copyediting is time consuming and unexciting; does anyone know of similar crowd sourced efforts? I found a few but most still had paid writers.

Comment author: ESRogs 04 April 2015 03:20:44AM 3 points [-]

I'd try 1n & 2n-gramming the contents against Google Ngrams

What does 1n or 2n-gramming mean? I'm looking at Google Ngrams, and it's not obvious to me.

Comment author: Persol 04 April 2015 10:24:06PM 9 points [-]

1 gramming is checking single words; should identify unfamiliar vocabulary. (Ex: quantifiable)

2 gramming would check pairs of words; should identify uncommon phrases made of common words (ex: probability mass - better examples probably exist)

The 1/2 gram terminology may be made up, but I think I've heard it used before.

Comment author: ESRogs 05 April 2015 03:36:26AM 2 points [-]

Thanks!

Comment author: Jayson_Virissimo 04 April 2015 01:01:50AM 3 points [-]

'Accessible Rationality' already exists... in the form of a wildly popular Harry Potter fanfiction.

Comment author: Kenny 04 April 2015 12:38:31AM 2 points [-]

What's the payoff of changing hyperlinks to footnotes? Given all of the other, substantive, issues you raised, that seems unlikely to make any significant difference.

Comment author: Persol 04 April 2015 01:00:34AM 7 points [-]

Two reason:

  • Frequently having multiple words as hyperlinks in ebooks mean that 'turning the page' may instead change chapters. Maybe it is just a problem with iPhone kindle.
  • For links that reference forward chapters, what is a new reader to do? They can ignore it and not understand the reference, or they can click, read, and then try to go back... but it's not a very smooth reading experience.

Granted, I probably wouldn't have noticed the second issue, if not for the first issue.

Comment author: ChristianKl 01 April 2015 07:46:44PM 1 point [-]

I don't think the point of the sequences or the book is to be accessible to everyone. If you want to write 'Accessible Rationality' it likely makes more sense to start from stretch.

Comment author: Persol 01 April 2015 09:23:21PM 5 points [-]

Agreed that it may not be the point, but other than what I think are fixable issues, the book contents work well. I don't think starting from scratch would be a large enough improvement to justify the extra time and increased chance of failure.

I think the big work is in making the examples accessible, and Eliezer already did this for the -other- negative trigger.

"If you want to make a point about science, or rationality, then my advice is to not choose a domain from contemporary politics if you can possibly avoid it. If your point is inherently about politics, then talk about Louis XVI during the French Revolution. Politics is an important domain to which we should individually apply our rationality— but it’s a terrible domain in which to learn. Why would anyone pick such a distracting example to illustrate nonmonotonic reasoning?"

Comment author: alexvermeer 13 March 2015 04:46:10PM 13 points [-]

Just a reminder that mistakes/problems/errors can be sent to errata@intelligence.org and we'll try fix them!

Comment author: quinox 15 March 2015 10:52:47AM 10 points [-]

I can't mail that address, I get a failure message from Google:

We're writing to let you know that the group you tried to contact (errata) may not exist, or you may not have permission to post messages to the group.

I'll post my feedback here:

Hello,

I got the book "Rationality: From AI to Zombies" via intelligence.org/e-junkie for my Kindle (5th gen, not the paperwhite/touch/fire). So far I've read a dozen pages, but since it will take me a while to get to the end of the book I'll give some feedback right away:

  • The book looks great! Some other ebooks I have don't use page-breaks at the end of a chapter, don't have a Table of Content, have inconsistent font types/sizes etc. The PDF version is very pretty as well.

  • The filename "Rationality.mobi" (AI-Zombie) is the same as "rationality.mobi" (HPMOR)

  • A bunch of inter-book links such as "The Twelve Virtues of Rationality"/"Predictably Wrong"/"Fake Beliefs"/"Noticing Confusion" (all from Biases: An introduction) don't work: On my Kindle I have the option to "Follow link", but when I choose it the page refreshes and I'm still at the same spot.

    Inspecting the .mobi source with Calibre e-book reader I see:

    < a href="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" >Noticing Confusion< /a>

    The links from the TOCs and some other chapters do work properly.

  • Due to the lack of quotation marks and the nonuse of italics I didn't realize the part "As a consequence, it might be necessary" was a quote (Biases: An introduction). The extra margin left and right do indicate something special, but with the experience of so many bad ebooks my brain assumed it was just a broken indentation level.

  • The difference between a link going to the web and one going to a location within the book aren't obvious: one is only a slighter darker grey than the other. In Calibri the links are a nice green/blue, but my kindle doesn't have colours.

Cheers,

Comment author: lukeprog 15 March 2015 06:27:14PM 5 points [-]

I can't mail that address, I get a failure message from Google

Oops. Should be fixed now.

Comment author: alexvermeer 16 March 2015 04:11:09PM 3 points [-]

The book looks great!

Thanks!

A bunch of inter-book links such as ...

D'oh. It's all good in the epub, but something broke (for very dumb reasons) converting the mobi. It's fixed now. If you've already bought the book though Amazon or e-junkie, you'll have to re-download the file to get the fixed one (in a few hours, while Amazon approves the new book). Sorry about that.

The difference between a link going to the web and one going to a location within the book aren't obvious: one is only a slighter darker grey than the other. In Calibri the links are a nice green/blue, but my kindle doesn't have colours.

Not much we can do about this. Amazon is very restrictive in how you can modify the styling of links. It works fine for displays with color, but people with e-ink displays are out of luck. :-(

Thanks.

Comment author: adamzerner 15 March 2015 01:58:33PM 0 points [-]

Same.

Comment author: iarwain1 13 March 2015 04:57:33PM *  10 points [-]

we'll try fix them

I think you meant "try to fix them" :)

Comment author: pinyaka 13 March 2015 05:14:42PM 24 points [-]

You should send that to errata@intelligence.org.

Comment author: Benito 13 March 2015 07:12:11AM 10 points [-]

Yay! Now I'm sending this to all of my friends!

Comment author: Caue 13 March 2015 05:46:12PM *  7 points [-]

My first reaction as well.

But that is easy. What I haven't figured out yet is how to get them to read it.

Comment author: Benito 13 March 2015 06:15:24PM *  4 points [-]

I've found that the people most interested in reading it are the ones I've already gotten addicted to HPMOR.

Comment author: bramflakes 13 March 2015 11:13:36PM *  9 points [-]

One of the most common complaints about the old Sequences was that there was no canonical default order, especially for people who didn't want to read the entire blog archive chronologically.

I was tricked into doing this. Years ago someone posted an ebook claiming to be the Sequences, but was actually just every single Yudkowsky blog post from 2006 to 2010 -_-

It took until noticing that only Yudkowsky's side of the FOOM debate was in there that I realized what had happened

Comment author: Vulture 16 March 2015 01:28:59AM 5 points [-]

Just as a little bit of a counterpoint, I loved the 2006-2010 ebook and was never particularly bothered by the length. I read the whole thing at least twice through, I think, and have occasionally used it to look up posts and so on. The format just worked really well for me. This may be because I am an unusually fast reader, or because I was young and had nothing else to do. But it certainly isn't totally useless :P

Comment author: bramflakes 16 March 2015 11:20:08PM *  4 points [-]

Oh, I didn't mean to imply I didn't like it! It was a welcome companion for hundreds of long school bus journeys.

Comment author: ciphergoth 14 March 2015 01:08:54PM 5 points [-]

It wasn't meant as a trick! Organising them would have been very hard.

Comment author: RobbBB 15 March 2015 05:52:06AM 6 points [-]

Can confirm!

Comment author: slicko 13 March 2015 06:47:50PM *  9 points [-]

Good work guys!

This might be the excuse I need to finally go through the complete sequences as opposed to relying on cherry-picking posts whenever I encounter a reference I don't already know.

Comment author: [deleted] 14 March 2015 07:48:37PM *  8 points [-]

I paid $0 because I'd rather not pay transaction fees on a donation to charity. You can donate to MIRI directly here:

https://intelligence.org/donate/

And CFAR here:

http://rationality.org/donate/

Comment author: malo 15 March 2015 04:19:47PM 3 points [-]

See my comment here about this.

Comment author: [deleted] 15 March 2015 05:25:20PM 2 points [-]

I used and prefer Bitcoin, which wasn't an option for the eBook and which carries smaller fees.

Comment author: Coscott 13 March 2015 03:17:28PM *  22 points [-]

The cover is incorrect :(

EDIT: If you do not understand this post, read essay 268 from the book!

Comment author: B_For_Bandana 13 March 2015 09:20:03PM 11 points [-]

The code of the shepherds is terrible and stern. One sheep, one pebble, hang the consequences. They have been known to commit fifteen, and twenty-one, and even even, rather than break it.

Comment author: Error 13 March 2015 06:19:18PM 4 points [-]

I just bust out laughing in the office at this...and can't share the joke with anybody.

Now I want to know if the incorrectness is intentional and if so, what message it's supposed to carry.

Comment author: lmm 17 March 2015 10:58:29PM 2 points [-]

It's a bluff to make us think Yudkowsky cares about things like human happiness rather than what's right. Don't be fooled!

Comment author: ciphergoth 13 March 2015 04:19:37PM 4 points [-]
Comment author: Quill_McGee 13 March 2015 06:30:11PM 1 point [-]

There might be one more stone not visible?

Comment author: Transfuturist 13 March 2015 06:38:17PM *  1 point [-]

I see nine stones, not ten.

Comment author: ciphergoth 14 March 2015 01:08:10PM 1 point [-]

Three at the back, three at the front, one to one side, one standing up... the question is whether it's standing on one stone or two.

Comment author: [deleted] 14 March 2015 07:45:38PM 1 point [-]

10 would still be incorrect.

Comment author: Quill_McGee 19 March 2015 11:51:19PM 1 point [-]

Darn it, and I counted like five times to make sure there really were 10 visible before I said anything. I didn't realize that the stone the middle-top stone was on top of was one stone, not two.

Comment author: spxtr 16 March 2015 07:49:42AM 7 points [-]

I am impressed. The production quality on this is excellent, and the new introduction by Rob Bensinger is approachable for new readers. I will definitely be recommending this over the version on this site.

Comment author: Ixiel 14 March 2015 12:03:57PM 7 points [-]

Excellent, thank you! Any update on when the real book will be available for purchase for those of us who don't do ebooks?

Comment author: Ivan_Tishchenko 20 October 2015 11:32:12AM 1 point [-]

I second this question! I want to have this book in flesh, staying on my bookshelf.

Comment author: kotrfa 13 March 2015 04:24:34PM *  7 points [-]

Can I know to who and where the money for the book goes?

Comment author: alexvermeer 13 March 2015 04:40:17PM 13 points [-]

From Amazon, 30% goes to Amazon and 70% goes to MIRI.

From e-junkie (the pay-what-you-want option): 100% goes to MIRI, minus PayPal transaction fees (a few %).

Comment author: DanielLC 14 March 2015 02:25:19AM 7 points [-]

Couldn't you pay $0.00, send the money to MIRI, and avoid transaction fees?

Comment author: RobbBB 15 March 2015 05:55:49AM 7 points [-]

Yeah. Main reason to do it this way is fear of trivial inconveniences.

Comment author: malo 15 March 2015 04:17:38PM 3 points [-]

Depending on how you sent money to MIRI, we'd incur transaction fees anyway (donating through PayPal using a PayPal account or CC). ACH donations have lower fees, and checks don't have any, but both of those take staff time to process, so unless the donation was say $50 or more, it probably wouldn't be worth it.

Comment author: [deleted] 15 March 2015 05:26:18PM 1 point [-]

What about Bitcoin?

Comment author: malo 15 March 2015 11:56:30PM 5 points [-]

No fees, but also takes some extra staff time (additional bookkeeping/accounting work is involved), so there is some cost to it. If we got more BTC donations it would reduce the time cost per donation, due to effects of batching, but as it stands now, they are usually processed (record added to our donor database and accounting software) on an individual basis.

One thing that takes a significant amount of time is when someone mis-pays a Coinbase invoice (sends a different amount of BTC then they indicated on the Coinbase form on our site). Coinbase treats these payments in a different way that ends up requiring more time to process on our end.

All that being said we like having the BTC donation option, and it always makes me happy to see one come in. So if making contributions via BTC is your preference, I'm all for it :)

Comment author: ike 15 March 2015 06:21:31PM 1 point [-]

They use coinbase, so according to this it's free up to $1 million.

Comment author: [deleted] 15 March 2015 07:29:12PM 1 point [-]

It should be free, period. Coinbase doesn't charge fees for registered non-for-profits.

Comment author: alexvermeer 14 March 2015 03:38:23AM 1 point [-]

Yup, but those are convenient distribution platforms.

Comment author: adamzerner 15 March 2015 01:32:04PM 3 points [-]

Perhaps this should be noted in the main article. I was thinking about buying it through Amazon until I saw this!

Comment author: MarkusRamikin 01 April 2015 08:56:54AM *  6 points [-]

For reasons, I suggest that Bayesian Judo doesn't make EY look good to people who aren't already cheering for his team, and maybe it wasn't wise to include it.

More generally, the book feels a bit... neutered. Things like, for example, changing "if you go ahead and mess around with Wulky's teenage daughter" to "if you go ahead and insult Wulky". The first is concrete, evocative, and therefore strong, while the latter is fuzzy and weak. Though my impression may be skewed just because I remember the original examples so well.

Comment author: Mirzhan_Irkegulov 01 May 2016 10:02:02PM 2 points [-]

My opinion: remove Bayesian Judo and add Whining-Based Communities. Seriously, Whining-Based Communities is the most powerful article I've ever read on LW, it symbolizes what rationality is about most of all. The point of rationality is achieving your goals despite cognitive biases, signaling, self-delusion, mysterious answers etc. It's very easy to brainwash yourself into thinking that you are “doing a good job”. It's very hard to put extra effort into actually doing what is the most effective, because it might go against your habits, self-image, intuitions, convictions etc.

Comment author: ilzolende 15 March 2015 10:28:34PM 6 points [-]

I am thinking of recommending this to people, all of whom are unlikely to pay. Is having people acquire this for $0 who would otherwise not have read it beneficial or harmful to MIRI? (If the answer is "harmful because of paying for people to download it", I can email it to my friends with a payment link instead of directing them to your website.)

Comment author: malo 15 March 2015 11:59:36PM *  7 points [-]

Definitely beneficial, there is no cost worth considering when it comes to the next marginal person getting the book through our site, even if their selection is $0. So don't worry about directing them there.

Comment author: imuli 13 March 2015 06:45:15PM 6 points [-]

The zip file has some extra Apple metadata files included. Nothing too revealing, just dropbox bits.

Comment author: Kaj_Sotala 18 March 2015 09:15:26AM *  5 points [-]

I liked Robby's introduction to the book overall, but I find it somewhat ironic that right after the prologue where Eliezer mentions that one of his biggest mistakes in writing the Sequences was focusing on abstract philosophical problems that are removed from people's daily problems, the introduction begins with

Imagine reaching into an urn that contains seventy white balls and thirty red ones, and plucking out ten mystery balls.

The first (though not necessarily best) example of how to rewrite this in less abstract form that comes to mind would be something like "Imagine that you're standing by the entrance of a university whose students are seven tenths female and three tenths male, and observing ten students go in..."; with the biased example being "On the other hand, suppose that you happen to be standing by the entrance of the physics department, which is mostly male even though the university in general is mostly female."

Some unnecessary technical jargon that could have been gotten rid of also caught my eye in the first actual post: e.g. "Rational agents make decisions that maximize the probabilistic expectation of a coherent utility function" could have been rewritten to be more broadly understandable, e.g. "rational agents make decisions that are the most likely to produce the kinds of outcomes they'd like to see".

I could spend some time making notes of these kinds of things and offering suggested rewrites for making the printed book more broadly accessible - would MIRI be interested in that, or would they prefer to keep the content as is?

Comment author: RobbBB 19 March 2015 12:33:35AM *  5 points [-]

Part of the idea behind the introduction is to replace an early series of posts: "Statistical Bias", "Inductive Bias", and Priors as Mathematical Objects. These get alluded to various times later in the sequences, and the posts 'An Especially Elegant Evolutionary Psychology Project', 'Where Recursive Justification Hits Bottom', and 'No Universally Compelling Arguments' all call back to the urn example. That said, I do think a more interesting example (whether or not it's more 'ordinary' and everyday) would be a better note to start the book on.

Do feel free to send stylistic or substantive change ideas to errata@intelligence.org, not just spelling errors.

Comment author: Gram_Stone 18 March 2015 09:30:37AM *  2 points [-]

This came to mind for me as well. This, from Burdensome Details, popped out at me: "Moreover, they would need to add absurdities—where the absurdity is the log probability, so you can add it—rather than averaging them." All this does for me is pattern-match to a Wikipedia article I once read about the concept of entropy in information theory; I don't really know what it means in any precise sense or why it might be true. And the essay even seems to stand on its own without that part. I've come to ignore my fear of not understanding things unless I don't understand pretty much everything I'm reading, but I think a lot of people would get scared that they didn't know enough to read the book and just stop reading.

Comment author: Kaj_Sotala 18 March 2015 10:16:57PM 5 points [-]

Come to think of it, we could collect proposed rewrites / deletions to some wiki page: this seems suitable for a communal effort. The "deletions" wouldn't actually need to be literal deletions, they could just be moved into a footnote. E.g. in the Burdensome Details article, a footnote saying something like "technically, you can measure probabilities by logarithms and..."

Comment author: RobbBB 19 March 2015 12:37:21AM *  4 points [-]

I like the idea of turning a lot of these jargony asides, especially early in the book, into footnotes. We'll be needing to make heavier use of footnotes anyway in order to explicitly direct people to other parts of the series in places where there will no longer be a clickable link. (Though we won't do this for most clickable links, just for the especially interesting / important ones.)

You're welcome to use a wiki page to list suggested changes, or a Google Doc; or just send a bunch of e-mails to errata@intelligence.org with ideas.

Comment author: Squark 13 March 2015 08:32:43AM *  5 points [-]

Congratulations, well done!

Side note: the "Glossary" link seems to be broken.

Comment author: RobbBB 13 March 2015 12:18:47PM 3 points [-]

Should be working now. I accidentally made it an internal link.

Comment author: Gust 19 March 2015 06:03:39AM *  4 points [-]

Hi, and thanks for the awesome job! Will you keep a public record of changes you make to the book? I'm coordinating a translation effort, and that would be important to keep it in sync if you change the actual text, not just fix spelling and hyperlinking errors.

Edit: Our translation effort is for Portuguese only, and can be found at http://racionalidade.com.br/wiki .

Comment author: RobbBB 19 March 2015 06:15:53AM *  4 points [-]

Yes, we'll keep a public record of content changes, or at least a private record that we'd be happy to share with people doing things like translation projects.

Comment author: hydkyll 12 April 2015 03:09:54PM 1 point [-]

How is that translation coming along? I could help with German.

Comment author: Gust 13 April 2015 02:44:16PM *  0 points [-]

We're translating to Brazilian Protuguese only, since that's our native language.

Comment author: Rangi 13 March 2015 04:33:00PM 4 points [-]

With SumatraPDF 3.0 on Windows 8.1 x64, the links in the PDF version do not show up. With Adobe Reader 11 on Windows 7 x86, they look fine. On the other hand, SumatraPDF can also handle the MOBI and EPUB versions.

Comment author: bramflakes 14 March 2015 12:08:28AM *  5 points [-]

I'm getting problems too. The contents pages look like this, for example.

Comment author: [deleted] 13 March 2015 09:43:35AM 4 points [-]

Don't have paypal or credit card or bitcoins or similar stuff, 0 price for now, I will look into donating from my Maestro debit card or maybe a direct transfer although international transfer rates may make that not worth the while. That and cash are the only methods I use - I rarely need anything I cannot buy with them. (I use gift cards purchased in shops for steam and google play.) I am thinking about purchasing some bitcoins for € for such donations purpose, if anyone can recommend a safe and debit card (or sofort.com) compatible service?

Comment author: MrMind 13 March 2015 08:47:01AM 3 points [-]

That's awesome!

Comment author: MrMind 13 March 2015 08:49:10AM 4 points [-]

Both links don't work though: you have lesswrong.com/ prefixing every correct address.

Comment author: RobbBB 13 March 2015 12:11:25PM 2 points [-]

Fixed! Thanks.

Comment author: Wind 13 August 2016 07:09:34PM 2 points [-]

I just finished listening to the Audiobook version of Rationality: From AI to Zombie. Lots of thanks to Yudkowsky and everyone else that was involved in making this book and the audio book. I do not know who the reader of the audio book is, but thanks all the same.

I am writing this comment as my way of prizing this book. I will try to summarize what I have personalty learned from it, in the hope that someone who was involved, will read this post and fell some pride in having helped me in my self improvement. But I am also writing this comment because I just want to express my thoughts after finishing the book.

I have not have any major change of mind, but I have several minor ones, which might very well continue to grow.

Listening to Yudkowsky's words have made me more confident, because he is saying many things that I already intuitively knew, but I could not properly explain it my self, and could therefore not be sure I was right. I am still not 100% certain I am right, but I am more confident, and I believe that this is a good thing. Smart people should be confident. No, this is not hind site bias, because:

  • I did not allays instantly agree, so I do know the difference.
  • I been actively introspecting since I was 12, so I know most of my brains tricks.

I never set out to be a rationalist. I don't even remember having a pre-LessWrong concept for the word "rationalist". There where just, correct thinking and in-correct thinking, and obviously correct thinking is the way that systematically leads you to the truth, because how else would you measure correctness. Maybe this saved me from falling in to some of the rationalist tropes that Yudkovsky warns about. Or maybe I avoided them because I have read to little science fiction. Or maybe it was because I looked at these types of tropes and saw an author who kinged to the, obviously wrong, but warm and fussy, idea that every human has the same number of skill points.

I wonder who setts out to be rational, with out having something specific they need rationality for. Maybe the same kind of people that identifies as an atheist? I am an atheist, but I don't identify as such, because in my country, this is mostly a non issue.

I found LessWrong because my new boyfriend encouraged me to read here, and I actually got through the book, because I like audiobooks.

The pre-LessWrong me was a truth seeker, and as such, I though a lot about the way as applied to truth seeking. I had a crisis of faith, a several years a go, questioning the validity of science. But never really though about applying systematic reasoning to decision under uncertainty. When, in my past, I was confronted with a decision, which I did not know how to reason out, I used to deliberately hand over the decision to my feelings. Because, I reasoned, if I don't know what is right anyway, I might as well save me the fight of going against my impulses. I hope that I can use what I have learned here to do better.

An other thing I have realized is that I am such a pushover for perceived social norms. I have notice a significant mental shift in my brain, just from having some one in my ear, who casually mentions many words and cryonics, as if these where the most normal things in the world. Intellectually I was already convinced, I already knew the right answer before listening to the book, but I still needed the extra nagging, to get all of may brain on board with it. I think that this has been the single most important insight I got from the book.

One reason I have not tried to develop the art of rational decision making before, is that I knew that I was not strong enough to counter my emotional preferences. But I was wrong. I now have one, systematically applicable self hack, and probably there are more out there to find. I have hope to be able to take charge of my motivation, and I have reasons to fight for control.

Current me is an aspiring effective altruist. I do not strive to be a perfect altruist, because I do have some selfish preferences, that I do not expect to go away. But I am going to get my ass out of the comfortable bubble of I can''t do anything anyway, and do something. Though I have not decided yet if I am going to take the path of earn to give, or if I should get directly involved in some project my self. I am looking in to both ways.

Finlay, here is one my favorite quotes from the book:

I pause. “Well…” I say slowly. “Frankly, I’m not entirely sure myself where this ‘reality’ business comes from. I can’t create my own reality in the lab, so I must not understand it yet. But occasionally I believe strongly that something is going to happen, and then something else happens instead. I need a name for whatever-it-is that determines my experimental results, so I call it ‘reality’. This ‘reality’ is somehow separate from even my very best hypotheses. Even when I have a simple hypothesis, strongly supported by all the evidence I know, sometimes I’m still surprised. So I need different names for the thingies that determine my predictions and the thingy that determines my experimental results. I call the former thingies ‘belief’, and the latter thingy ‘reality’.”

Comment author: jrincayc 02 May 2016 07:03:07PM 2 points [-]

I have been creating a tex version at: https://github.com/jrincayc/rationality-ai-zombies

Comment author: jrincayc 27 August 2017 12:53:10PM 0 points [-]

I have relinked the footnotes. It is now reasonably editable. I've put up pdfs at https://github.com/jrincayc/rationality-ai-zombies/releases

Comment author: jrincayc 20 May 2016 02:55:27AM *  0 points [-]

There is still a lot of work to do before I consider it done, but it is more or less useable for some purposes. I printed off a copy for myself from Lulu for about $12. Here is the two column version that can be printed out as a single volume: http://jjc.freeshell.org/rationality-ai-zombies/rationality_from_ai_to_zombies_two_column_beta2.pdf

Comment author: John_Mitchell 12 June 2015 12:03:14PM 2 points [-]

Do people think there is value in making an audio book from this?

I was thinking it would be possible to do in a similar process to the HPMOR audiobook with people contributing different chapters. If there is interest in doing this and if it is permitted to be done then I will happily volunteer to coordinate the effort. If this idea does have support then given the discussion below about how the book could be improved, would it make more sense to postpone an audiobook to allow for sensible changes, or is that an unnecessary delay in search of unreachable perfection?

Comment author: Vaniver 12 June 2015 03:28:24PM 3 points [-]

Do people think there is value in making an audio book from this?

Yes; one is being made by Castify.

Comment author: [deleted] 02 April 2015 06:35:18AM 2 points [-]

Does the book (especially the printed version) have training problems after sections? (I don't have it, sorry if the question is redundant).

Comment author: [deleted] 02 April 2015 07:08:20AM 1 point [-]

It does not.

Comment author: [deleted] 02 April 2015 07:19:27AM 0 points [-]

Maybe it should, for people who won't discuss things online for some reason.

Comment author: Tenoke 13 March 2015 11:25:23AM 3 points [-]

Awesome! How large is it altogether (in words)?

Comment author: alexvermeer 13 March 2015 04:40:42PM 7 points [-]

Approximately 600,000 words!

Comment author: lukeprog 13 March 2015 09:48:28PM 11 points [-]

Which is roughly the length of War and Peace or Atlas Shrugged.

Comment author: Transfuturist 14 March 2015 05:53:15AM 5 points [-]

Ah, so about as large as it takes for a fanfic to be good. :P

Comment author: FiftyTwo 18 March 2015 09:06:49AM 1 point [-]

Might be worth including the Amazon.co.uk and other store links.

Comment author: Wind 15 August 2016 05:24:40PM 0 points [-]

A friend of mine is interested in reading this book, but would prefer a printed copy. Is there any chance that this book will be published any time soon?

Comment author: jrincayc 27 August 2017 12:58:09PM 0 points [-]

I have used the two column version: https://github.com/jrincayc/rationality-ai-zombies/releases/download/beta3/rationality_from_ai_to_zombies_2c.pdf with https://www.lulu.com/ to make a printed version for myself.

Note that there are problems with the that pdf, so it isn't perfect, but it might work. The regular PDF is too long to print as a single book.

Comment author: Mirzhan_Irkegulov 04 August 2015 10:56:27AM 0 points [-]

Is it possible to make the book available on Google Play Books? What might be reasons not to include the book there?

Comment author: xoda 14 April 2015 01:28:57AM *  0 points [-]

Sorry for my problem.I tried 15 times downloading,only once started and stopped at 1.5M/30.6M. Others can't even get to track. I wish to use another source ,or some kind friends could send the pdf to 513493106@qq.com?DEEPLY BOW for your help!