This is a follow-up to last year's report. Here, I will talk about my successes and failures using Spaced Repetition Software (SRS) in the classroom for a second year. The year's not over yet, but I have reasons for reporting early that should become clear in a subsequent post. A third post will then follow, and together these will constitute a small sequence exploring classroom SRS and the adjacent ideas that bubble up when I think deeply about teaching.

Summary

I experienced net negative progress this year in my efforts to improve classroom instruction via spaced repetition software. While this is mostly attributable to shifts in my personal priorities, I have also identified a number of additional failure modes for classroom SRS, as well as additional shortcomings of Anki for this use case. My experiences also showcase some fundamental challenges to teaching-in-general that SRS depressingly spotlights without being any less susceptible to. Regardless, I am more bullish than ever about the potential for classroom SRS, and will lay out a detailed vision for what it can be in the next post.

Note 1: I'm not very serious about the second part of the title, I just thought it sounds more catchy. I'm a long time lurker writing here for the first time, and it's not my intention to alienate anyone. Also, hi, nice to meet you. Please leave a comment to achieve a result of making me happy about you having left a comment. But let's get to the point.

I think you might be familiar with TED Talks. Recall the last time you watched one, and how you felt while doing it.

[BZRT BZRT sound of imagination working]

In my case, I often got the feeling like if I was learning something valuable while watching most TED Talks. The speakers are (mostly) obviously passionate and intelligent people, speaking about important matters they care about a lot. (Granted, I probably haven't watched more than a dozen TED Talks in all my life, so my sample is quite small, but I think it isn't very unrepresentative.)

But at some point, I started asking myself afterwards:

So, what have I actually learned?

Which translates in my internal dialect to:

For each major point, give a one-sentence summary and at least one example of how I could apply it.

(Note 2: don't treat this "one sentence summary" thing too strictly - of course it's only a reflex/shorthand that is useful in many situations, but not all. I like it because it's simple enough that it's installable as a subconscious trigger-action.)

And I could not state afterwards anything actually useful that I have learned from those "fascinating" videos (with at most one or two small exceptions).

This is exactly what I mean by "Education as Entertainment".

It's getting the enjoyable *feeling* of learning without any real progress.

[DUM DUM DUM sound of increasing dramatism]

And now, what if you use this concept to look at rationality materials?

For me, reading the core Eliezer's braindump (basically the content of "From AI to Zombies"), as well as braindumps (in the form of blogs) of several other people from the LW community, had definite learning value.

I take notes when I read those, and I have an accountability system in place that enables me to make sure I follow up on all the advice I give to myself, test the new ideas, and improve/drop/replace/implement as needed.

However, when I read (a significant part of) the content produced by the "modern" community-powered-LessWrong, I classify its actual learning value at around the same level as TED Talks.

Or YouTube videos with cats, only those don't give me the *impression* that I'm learning something.

THE END

Please let me know what you think.

Final Note: Please take my remarks with a grain of salt. What I write is meant to inspire thoughts in you, not to represent my best factual knowledge about the LW community.

UPDATE 3/16/16: I decided to go to public school, because I was tired of all the little annoying stuff at my current school--especially the entitled kids and the entitled attitude in general. Everybody acts like they deserve something. It's very irritating.

The other reason I came to that decision was "exploration value". By moving to a new situation I learn whether I really am better off in the kind of environment offered by the public high school; even if it ends up being worse for me, at least I know what to avoid. If it's good, I know it's good; and if it makes no difference, I know that, too.

EDIT: Just to clarify, I attended public middle school with the same group of kids, so I won't have to worry too much about getting to know new people. I am still in occasional contact with my old friends. I talked to one of them for several hours just yesterday.

(I'm new here, though I've lurked, so if I break any rules or otherwise do something detrimental, please let me know and I will try to correct my mistake)

I currently go to a rather nice independent high school. I'm on significant financial aid, so I can afford it. The academics there are outstanding. However, being a boarding school (I go as a day student) it requests a lot of our free time. We are required to participate in adult-sanctioned activities at least six or seven hours a week, in addition to normal classes. This means that 1. I get home from school around 6pm and 2. (more importantly) it's very hard to socialize when you don't board at the school, and there's really very little besides drama or sports (neither of which I like very much) that people do after school and actually enjoy/make friends in.

I'm strongly considering transferring to my public school, which is unusually good for a public school, as a junior next year. The academics are not as great (classes are less discussion-based and there are not as many APs offered) but there is a strong amount of participation in stuff I might actually enjoy after school (math team, etc.) because we're not required to do anything after school. I've noticed that when people tell me I have to do things, I enjoy them much, much less. Also, I won't have a commute, which would be nice.

Everything else about the schools is more or less comparable.

I'm sure when I think about this decision I am biased in some way. I'm probably succumbing to the sunk cost fallacy (sometimes I think if I'd been at the public school the last two years, I'd rather be at the private school) or something like that. If not, I'm facing the problem of Buridan's Ass.

My question to you: should I transfer or not? I have thought very hard and consulted several intelligent people, and have not been able to come to any sort of conclusion.

I wonder if anyone has thought about setting up an online community dedicated to peer-to-peer tutoring.  The idea is that if I want to learn "Differential Geometry" and know "Python programming", and you want to learn "Python programming" and know "Differential geometry," then we can agree to tutor each other online.  The features of the community would be to support peer-to-peer tutoring by:

• Facilitating matchups between compatible tutors
• Allowing for more than two people to participate in a tutoring arrangement
• Providing reputation-based incentives to honor tutoring agreements and putting effort into tutoring
• Allowing other members to "sit in" on tutoring sessions, if they are made public
• Allowing the option to record tutoring sessions
• Providing members with access to such recorded sessions and "course materials"
• Providing a forum to arrange other events

I made a series of game theory videos that carefully go through the mechanics of solving many different types of games.  I optimized the videos for my future Smith College game theory students who will either miss a class, or get lost in class and want more examples.   I emphasize clarity over excitement.   I would be grateful for any feedback.

Last year, I asked LW for some advice about spaced repetition software (SRS) that might be useful to me as a high school teacher. With said advice came a request to write a follow-up after I had accumulated some experience using SRS in the classroom. This is my report.

Please note that this was not a scientific experiment to determine whether SRS "works." Prior studies are already pretty convincing on this point and I couldn't think of a practical way to run a control group or "blind" myself. What follows is more of an informal debriefing for how I used SRS during the 2014-15 school year, my insights for others who might want to try it, and how the experience is changing how I teach.

Summary

SRS can raise student achievement even with students who won't use the software on their own, and even with frequent disruptions to the study schedule. Gains are most apparent with the already high-performing students, but are also meaningful for the lowest students. Deliberate efforts are needed to get student buy-in, and getting the most out of SRS may require changes in course design.

The software

After looking into various programs, including the game-like Memrise, and even writing my own simple SRS, I ultimately went with Anki for its multi-platform availability, cloud sync, and ease-of-use. I also wanted a program that could act as an impromptu catch-all bin for the 2,000+ cards I would be producing on the fly throughout the year. (Memrise, in contrast, really needs clearly defined units packaged in advance).

The students

I teach 9th and 10th grade English at an above-average suburban American public high school in a below-average state. Mine are the lower "required level" students at a school with high enrollment in honors and Advanced Placement classes. Generally speaking, this means my students are mostly not self-motivated, are only very weakly motivated by grades, and will not do anything school-related outside of class no matter how much it would be in their interest to do so. There are, of course, plenty of exceptions, and my students span an extremely wide range of ability and apathy levels.

The procedure

First, what I did not do. I did not make Anki decks, assign them to my students to study independently, and then quiz them on the content. With honors classes I taught in previous years I think that might have worked, but I know my current students too well. Only about 10% of them would have done it, and the rest would have blamed me for their failing grades—with some justification, in my opinion.

Instead, we did Anki together, as a class, nearly every day.

As initial setup, I created a separate Anki profile for each class period. With a third-party add-on for Anki called Zoom, I enlarged the display font sizes to be clearly legible on the interactive whiteboard at the front of my room.

Nightly, I wrote up cards to reinforce new material and integrated them into the deck in time for the next day's classes. This averaged about 7 new cards per lesson period.These cards came in many varieties, but the three main types were:

1. concepts and terms, often with reversed companion cards, sometimes supplemented with "what is this an example of" scenario cards.
2. vocabulary, 3 cards per word: word/def, reverse, and fill-in-the-blank example sentence
3. grammar, usually in the form of "What change(s), if any, does this sentence need?" Alternative cards had different permutations of the sentence.

Weekly, I updated the deck to the cloud for self-motivated students wishing to study on their own.

Daily, I led each class in an Anki review of new and due cards for an average of 8 minutes per study day, usually as our first activity, at a rate of about 3.5 cards per minute. As each card appeared on the interactive whiteboard, I would read it out loud while students willing to share the answer raised their hands. Depending on the card, I might offer additional time to think before calling on someone to answer. Depending on their answer, and my impressions of the class as a whole, I might elaborate or offer some reminders, mnemonics, etc. I would then quickly poll the class on how they felt about the card by having them show a color by way of a small piece of card-stock divided into green, red, yellow, and white quadrants. Based on my own judgment (informed only partly by the poll), I would choose and press a response button in Anki, determining when we should see that card again.

[Data shown is from one of my five classes. We didn't start using Anki until a couple weeks into the school year.]

Opportunity costs

8 minutes is a significant portion of a 55 minute class period, especially for a teacher like me who fills every one of those minutes. Something had to give. For me, I entirely cut some varieties of written vocab reinforcement, and reduced the time we spent playing the team-based vocab/term review game I wrote for our interactive whiteboards some years ago. To a lesser extent, I also cut back on some oral reading comprehension spot-checks that accompany my whole-class reading sessions. On balance, I think Anki was a much better way to spend the time, but it's complicated. Keep reading.

Whole-class SRS not ideal

Every student is different, and would get the most out of having a personal Anki profile determine when they should see each card. Also, most individuals could study many more cards per minute on their own than we averaged doing it together. (To be fair, a small handful of my students did use the software independently, judging from Ankiweb download stats)

Getting student buy-in

Before we started using SRS I tried to sell my students on it with a heartfelt, over-prepared 20 minute presentation on how it works and the superpowers to be gained from it. It might have been a waste of time. It might have changed someone's life. Hard to say.

As for the daily class review, I induced engagement partly through participation points that were part of the final semester grade, and which students knew I tracked closely. Raising a hand could earn a kind of bonus currency, but was never required—unlike looking up front and showing colors during polls, which I insisted on. When I thought students were just reflexively holding up the same color and zoning out, I would sometimes spot check them on the last card we did and penalize them if warranted.

But because I know my students are not strongly motivated by grades, I think the most important influence was my attitude. I made it a point to really turn up the charm during review and play the part of the engaging game show host. Positive feedback. Coaxing out the lurkers. Keeping that energy up. Being ready to kill and joke about bad cards. Reminding classes how awesome they did on tests and assignments because they knew their Anki stuff.

(This is a good time to point out that the average review time per class period stabilized at about 8 minutes because I tried to end reviews before student engagement tapered off too much, which typically started happening at around the 6-7 minute mark. Occasional short end-of-class reviews mostly account for the difference.)

I also got my students more on the Anki bandwagon by showing them how this was directly linked reduced note-taking requirements. If I could trust that they would remember something through Anki alone, why waste time waiting for them to write it down? They were unlikely to study from those notes anyway. And if they aren't looking down at their paper, they'll be paying more attention to me. I better come up with more cool things to tell them!

Making memories

Everything I had read about spaced repetition suggested it was a great reinforcement tool but not a good way to introduce new material. With that in mind, I tried hard to find or create memorable images, examples, mnemonics, and anecdotes that my Anki cards could become hooks for, and to get those cards into circulation as soon as possible. I even gave this method a mantra: "vivid memory, card ready".

When a student during review raised their hand, gave me a pained look, and said, "like that time when...." or "I can see that picture of..." as they struggled to remember, I knew I had done well. (And I would always wait a moment, because they would usually get it.)

Baby cards need immediate love

Unfortunately, if the card wasn't introduced quickly enough—within a day or two of the lesson—the entire memory often vanished and had to be recreated, killing the momentum of our review. This happened far too often—not because I didn't write the card soon enough (I stayed really on top of that), but because it didn't always come up for study soon enough. There were a few reasons for this:

1. We often had too many due cards to get through in one session, and by default Anki puts new cards behind due ones.
2. By default, Anki only introduces 20 new cards in one session (I soon uncapped this).
3. Some cards were in categories that I gave lower priority to.

Two obvious cures for this problem:

1. Make fewer cards. (I did get more selective as the year went on.)
2. Have all cards prepped ahead of time and introduce new ones at the end of the class period they go with. (For practical reasons, not the least of which was the fact that I didn't always know what cards I was making until after the lesson, I did not do this. I might able to next year.)

Days off suck

SRS is meant to be used every day. When you take weekends off, you get a backlog of due cards. Not only do my students take every weekend and major holiday off (slackers), they have a few 1-2 week vacations built into the calendar. Coming back from a week's vacation means a 9-day backlog (due to the weekends bookending it). There's no good workaround for students that won't study on their own. The best I could do was run longer or multiple Anki sessions on return days to try catch up with the backlog. It wasn't enough. The "caught up" condition was not normal for most classes at most points during the year, but rather something to aspire to and occasionally applaud ourselves for reaching. Some cards spent weeks or months on the bottom of the stack. Memories died. Baby cards emerged stillborn. Learning was lost.

Needless to say, the last weeks of the school year also had a certain silliness to them. When the class will never see the card again, it doesn't matter whether I push the button that says 11 days or the one that says 8 months. (So I reduced polling and accelerated our cards/minute rate.)

Never before SRS did I fully appreciate the loss of learning that must happen every summer break.

Triage

I kept each course's master deck divided into a few large subdecks. This was initially for organizational reasons, but I eventually started using it as a prioritizing tool. This happened after a curse-worthy discovery: if you tell Anki to review a deck made from subdecks, due cards from subdecks higher up in the stack are shown before cards from decks listed below, no matter how overdue they might be. From that point, on days when we were backlogged (most days) I would specifically review the concept/terminology subdeck for the current semester before any other subdecks, as these were my highest priority.

On a couple of occasions, I also used Anki's study deck tools to create temporary decks of especially high-priority cards.

Seizing those moments

Veteran teachers start acquiring a sense of when it might be a good time to go off book and teach something that isn't in the unit, and maybe not even in the curriculum. Maybe it's teaching exactly the right word to describe a vivid situation you're reading about, or maybe it's advice on what to do in a certain type of emergency that nearly happened. As the year progressed, I found myself humoring my instincts more often because of a new confidence that I can turn an impressionable moment into a strong memory and lock it down with a new Anki card. I don't even care if it will ever be on a test. This insight has me questioning a great deal of what I thought knew about organizing a curriculum. And I like it.

A lifeline for low performers

An accidental discovery came from having written some cards that were, it was immediately obvious to me, much too easy. I was embarrassed to even be reading them out loud. Then I saw which hands were coming up.

In any class you'll get some small number of extremely low performers who never seem to be doing anything that we're doing, and, when confronted, deny that they have any ability whatsoever. Some of the hands I was seeing were attached to these students. And you better believe I called on them.

It turns out that easy cards are really important because they can give wins to students who desperately need them. Knowing a 6th grade level card in a 10th grade class is no great achievement, of course, but the action takes what had been negative morale and nudges it upward. And it can trend. I can build on it. A few of these students started making Anki the thing they did in class, even if they ignored everything else. I can confidently name one student I'm sure passed my class only because of Anki. Don't get me wrong—he just barely passed. Most cards remained over his head. Anki was no miracle cure here, but it gave him and I something to work with that we didn't have when he failed my class the year before.

A springboard for high achievers

It's not even fair. The lowest students got something important out of Anki, but the highest achievers drank it up and used it for rocket fuel. When people ask who's widening the achievement gap, I guess I get to raise my hand now.

I refuse to feel bad for this. Smart kids are badly underserved in American public schools thanks to policies that encourage staff to focus on that slice of students near (but not at) the bottom—the ones who might just barely be able to pass the state test, given enough attention.

Where my bright students might have been used to high Bs and low As on tests, they were now breaking my scales. You could see it in the multiple choice, but it was most obvious in their writing: they were skillfully working in terminology at an unprecedented rate, and making way more attempts to use new vocabulary—attempts that were, for the most part, successful.

Given the seemingly objective nature of Anki it might seem counterintuitive that the benefits would be more obvious in writing than in multiple choice, but it actually makes sense when I consider that even without SRS these students probably would have known the terms and the vocab well enough to get multiple choice questions right, but might have lacked the confidence to use them on their own initiative. Anki gave them that extra confidence.

A wash for the apathetic middle?

I'm confident that about a third of my students got very little out of our Anki review. They were either really good at faking involvement while they zoned out, or didn't even try to pretend and just took the hit to their participation grade day after day, no matter what I did or who I contacted.

These weren't even necessarily failing students—just the apathetic middle that's smart enough to remember some fraction of what they hear and regurgitate some fraction of that at the appropriate times. Review of any kind holds no interest for them. It's a rerun. They don't really know the material, but they tell themselves that they do, and they don't care if they're wrong.

On the one hand, these students are no worse off with Anki than they would have been with with the activities it replaced, and nobody cries when average kids get average grades. On the other hand, I'm not ok with this... but so far I don't like any of my ideas for what to do about it.

Putting up numbers: a case study

For unplanned reasons, I taught a unit at the start of a quarter that I didn't formally test them on until the end of said quarter. Historically, this would have been a disaster. In this case, it worked out well. For five weeks, Anki was the only ongoing exposure they were getting to that unit, but it proved to be enough. Because I had given the same test as a pre-test early in the unit, I have some numbers to back it up. The test was all multiple choice, with two sections: the first was on general terminology and concepts related to the unit. The second was a much harder reading comprehension section.

As expected, scores did not go up much on the reading comprehension section. Overall reading levels are very difficult to boost in the short term and I would not expect any one unit or quarter to make a significant difference. The average score there rose by 4 percentage points, from 48 to 52%.

Scores in the terminology and concept section were more encouraging. For material we had not covered until after the pre-test, the average score rose by 22 percentage points, from 53 to 75%. No surprise there either, though; it's hard to say how much credit we should give to SRS for that.

But there were also a number of questions about material we had already covered before the pretest. Being the earliest material, I might have expected some degradation in performance on the second test. Instead, the already strong average score in that section rose by an additional 3 percentage points, from 82 to 85%. (These numbers are less reliable because of the smaller number of questions, but they tell me Anki at least "locked in" the older knowledge, and may have strengthened it.)

Some other time, I might try reserving a section of content that I teach before the pre-test but don't make any Anki cards for. This would give me a way to compare Anki to an alternative review exercise.

What about formal standardized tests?

I don't know yet. The scores aren't back. I'll probably be shown some "value added" analysis numbers at some point that tell me whether my students beat expectations, but I don't know how much that will tell me. My students were consistently beating expectations before Anki, and the state gave an entirely different test this year because of legislative changes. I'll go back and revise this paragraph if I learn anything useful.

Those discussions...

If I'm trying to acquire a new skill, one of the first things I try to do is listen to skilled practitioners of that skill talk about it to each other. What are the terms-of-art? How do they use them? What does this tell me about how they see their craft? Their shorthand is a treasure trove of crystallized concepts; once I can use it the same way they do, I find I'm working at a level of abstraction much closer to theirs.

Similarly, I was hoping Anki could help make my students more fluent in the subject-specific lexicon that helps you score well in analytical essays. After introducing a new term and making the Anki card for it, I made extra efforts to use it conversationally. I used to shy away from that because so many students would have forgotten it immediately and tuned me out for not making any sense. Not this year. Once we'd seen the card, I used the term freely, with only the occasional reminder of what it meant. I started using multiple terms in the same sentence. I started talking about writing and analysis the way my fellow experts do, and so invited them into that world.

Even though I was already seeing written evidence that some of my high performers had assimilated the lexicon, the high quality discussions of these same students caught me off guard. You see, I usually dread whole-class discussions with non-honors classes because good comments are so rare that I end up dejectedly spouting all the insights I had hoped they could find. But by the end of the year, my students had stepped up.

I think what happened here was, as with the writing, as much a boost in confidence as a boost in fluency. Whatever it was, they got into some good discussions where they used the terminology and built on it to say smarter stuff.

Don't get me wrong. Most of my students never got to that point. But on average even small groups without smart kids had a noticeably higher level of discourse than I am used to hearing when I break up the class for smaller discussions.

Limitations

SRS is inherently weak when it comes to the abstract and complex. No card I've devised enables a student to develop a distinctive authorial voice, or write essay openings that reveal just enough to make the reader curious. Yes, you can make cards about strategies for this sort of thing, but these were consistently my worst cards—the overly difficult "leeches" that I eventually suspended from my decks.

A less obvious limitation of SRS is that students with a very strong grasp of a concept often fail to apply that knowledge in more authentic situations. For instance, they may know perfectly well the difference between "there", "their", and "they're", but never pause to think carefully about whether they're using the right one in a sentence. I am very open to suggestions about how I might train my students' autonomous "System 1" brains to have "interrupts" for that sort of thing... or even just a reflex to go back and check after finishing a draft.

Moving forward

I absolutely intend to continue using SRS in the classroom. Here's what I intend to do differently this coming school year:

• Reduce the number of cards by about 20%, to maybe 850-950 for the year in a given course, mostly by reducing the number of variations on some overexposed concepts.
• Be more willing to add extra Anki study sessions to stay better caught-up with the deck, even if this means my lesson content doesn't line up with class periods as neatly.
• Be more willing to press the red button on cards we need to re-learn. I think I was too hesitant here because we were rarely caught up as it was.
• Rework underperforming cards to be simpler and more fun.
• Use more simple cloze deletion cards. I only had a few of these, but they worked better than I expected for structured idea sets like, "characteristics of a tragic hero".
• Take a less linear and more opportunistic approach to introducing terms and concepts.
• Allow for more impromptu discussions where we bring up older concepts in relevant situations and build on them.
• Shape more of my lessons around the "vivid memory, card ready" philosophy.
• Continue to reduce needless student note-taking.
• Keep a close eye on 10th grade students who had me for 9th grade last year. I wonder how much they retained over the summer, and I can't wait to see what a second year of SRS will do for them.

Suggestions and comments very welcome!

Right now, the inaugural class of Minerva Schools at KGI (part of the Claremont Colleges) is finishing up its first semester of college. I use the word "college" here loosely: there are no lecture halls, no libraries, no fraternities, no old stone buildings, no sports fields, no tenure... Furthermore, Minerva operates for profit (which may raise eyebrows), but appeals to a decidedly different demographic than DeVry etc; billed as the first "online Ivy", it relies on a proprietary online platform to apply pedagogical best practices. Has anyone heard of this before?

The Minerva Project's instructional innovations are what's really exciting. There are no lectures. There are no introductory classes. (There are MOOCs for that! "Do your freshman year at home.") Students meet for seminar-based online classes which are designed to inculcate "habits of mind"; professors use a live, interactive video platform to teach classes, which tracks students' progress and can individualize instruction. The seminars are active and intense; to quote from a recent (Sept. 2014) Atlantic article,

"The subject of the class ...was inductive reasoning. [The professor] began by polling us on our understanding of the reading, a Nature article about the sudden depletion of North Atlantic cod in the early 1990s. He asked us which of four possible interpretations of the article was the most accurate. In an ordinary undergraduate seminar, this might have been an occasion for timid silence... But the Minerva class extended no refuge for the timid, nor privilege for the garrulous. Within seconds, every student had to provide an answer, and [the professor] displayed our choices so that we could be called upon to defend them. [The professor] led the class like a benevolent dictator, subjecting us to pop quizzes, cold calls, and pedagogical tactics that during an in-the-flesh seminar would have taken precious minutes of class time to arrange."

It sounds to me like Minerva is actually making a solid effort to apply evidence-based instructional techniques that are rarely ever given a chance. There are boatloads of sound, reproducible experiments that tell us how people learn and what teachers can do to improve learning, but in practice they are almost wholly ignored. To take just one example, spaced repetition and the testing effect are built into the seminar platform: students have a pop quiz at the beginning of each class and another one at a random moment later in the class. Terrific! And since it's all computer-based, the software can keep track of student responses and represent the material at optimal intervals.

Also, much more emphasis is put on articulating positions and defending arguments, which is known to result in deeper processing of material. In general though, I really like how you are called out and held to account for your answers (again, from the Atlantic article:

...it was exhausting: a continuous period of forced engagement, with no relief in the form of time when my attention could flag or I could doodle in a notebook undetected. Instead, my focus was directed relentlessly by the platform, and because it looked like my professor and fellow edu-nauts were staring at me, I was reluctant to ever let my gaze stray from the screen... I felt my attention snapped back to the narrow issue at hand, because I had to answer a quiz question or articulate a position. I was forced, in effect, to learn.

Their approach to admissions is also interesting. The Founding Class had a 2.8% acceptance rate (a ton were enticed to apply on promise of a full scholarship) and features students from ~14 countries. In the application process, no consideration is given to diversity, balance of gender, or national origin, and SAT/ACT scores are not accepted: applicants must complete a battery of proprietary computer-based quizzes, essentially an in-house IQ test. If they perform well enough, they are invited for an interview, during which they must compose a short essay to ensure an authentic writing sample (i.e., no ghostwriters). After all is said and done, the top 30 applicants get in.

Anyway, I am a student and researcher in the field of educational psychology so this may not be as exciting to others. I'm surprised that I hadn't heard of it before though, and I'm really curious to see what comes of it!

One of many problems with the contemporary university system is that the same institutions that educate students also give them their degrees and grades. This obviously creates massive incentives for grade inflation and lowering of standards. Giving a thorough education requires hard work not only from students but also from the professors. In the absence of an independent body that tests that the students actually have learnt what they are supposed to have learnt, many professors spend as little time as possible at teaching, giving the students light workloads (something most of them of course happily accept). The faculty/student non-aggression pact is an apt term for this.

To see how absurd this system is, imagine that we would have the same system for drivers' licenses: that the driving schools that train prospective drivers also tested them and issued their drivers' licenses. In such a system, people would most probably chose the most lenient schools, leading to a lowering of standards. For fear of such a lowering of standards, prospective drivers are in many countries (I would guess universally but do not know that for sure) tested by government bodies.

Presumably, the main reason for this is that governments really care about the lowering of drivers' standards. Ensuring that all drivers are appropriately educated (i.e. is seen as very important. By contrast, the governments don't care that much about the lowering of academic standards. If they would, they would long ago have replaced a present grading/certification system with one where students are tested by independent bodies, rather than by the universities themselves.

This is all the more absurd given how much politicians in most countries talk about the importance of education. More often than not they talk about education, especially higher education, as a panacea to cure for all ills. However, if we look at the politicians' actions, rather than at their words, it doesn't seem like they actually do think it's quite as important as they say to ensure that the population is well-educated.

Changing the system for certifying students is important not the least in order to facilitate inventions in higher education. The present system discriminates in favour of traditional campus courses, which are both expensive and fail to teach the students as much as they should. I'm not saying that online courses, and other non-standard courses, are necessarily better or more cost-effective, but they should get the chance to prove that they are.

The system is of course hard to change, since there are lots of vested interests that don't want it to change. This is nicely illustrated by the reactions to a small baby-step towards the system that I'm envisioning that OECD is presently trying to take. Financial Times (which has a paywall, unfortunately) reports that OECD are attempting to introduce Pisa-style tests to compare students from higher education institutions around the world. Third year students would be tested on critical thinking, analytical reasoning, problem solving and written communcation. There would also be discipline-specific trials for economics and engineering.

These attempts have, however, not progressed because of resistance from some universities and member countries. OECD says that the resistance often comes from "the most prestigious institutions, because they have very little to win...and a lot to lose". In contrast, "the greatest supporters are the ones that add the greatest value...many of the second-tier institutes are actually a lot better and they're very keen to get on a level playing field."

I figure that if OECD get enough universities on board, they could start implementing the system without the obstructing top universities. They could also allow students from those universities to take the tests independently. If employers started taking these tests seriously, students would have every reason to take them even if their universities haven't joined. Slowly, these presumably more objective tests, or others like them, would become more important at the cost of the universities' inflated grades. People often try to change institutions or systems directly, but sometimes it is more efficient to build alternative systems, show that their useful to the relevant actors, and start out-competing the dominant system (as discussed in these comments).

A pattern of cognitive biases not yet discussed here are the biases due to having a narcissistic parent who seeks validation through the child’s academic achievements.

HPMOR clearly shows these biases: Harry's mother is narcissistic, impressed by education, and not particularly smart, and Harry does not realize how this affects his thinking.

Here is my evidence:

The Sorting Hat says Harry is driven by "the fear of losing your fantasy of greatness, of disappointing the people who believe in you" (Ch. 77). Psychology texts say that this fear is what children of a narcissistic parent usually feel. The child feels perpetually ignored because the narcissistic parent seeks validation from the child's accomplishments but refuses to actually listen to the child, spurring the child to ever greater heights of intellectual achievement. 

The text supports this view: “Always Harry had been encouraged to study whatever caught his attention, bought all the books that caught his fancy...given anything reasonable that he wanted, except, maybe, the slightest shred of respect” and “Petunia wrung her hands. She seemed to be on the verge of tears. "My love, I know I can't win arguments with you, but please, you have to trust me on this … I want my husband to, to listen to his wife who loves him, and trust her just this once - " (Ch. 1) describes a narcissistic, anxiously needy mother, an avoidant father, and a son whose parents provide for his physical needs but neglect his need for respect (ego). “If you conceived of yourself as a Good Parent, you would do it. But take a ten-year-old seriously? Hardly.” (Ch. 1) 

Harry goes Dark when the connection to his family is threatened. For example: "The black rage began to drain away, as it dawned on him that...his family wasn't in danger [of legal separation]" (ch. 5) indicates that Harry went Dark even though no one’s life was threatened. The cost of Harry’s Dark Side is becoming an adult at a young age: Harry says, “Every time I call on it... it uses up my childhood.” (Ch. 91). This is consistent with spending nearly all free time studying (instead of wasting time with friends) to impress Harry’s mother.

Typically, children of narcissistic parents inherit either narcissistic or people-pleasing traits. I predicted that if my theory is correct then Harry would have a narcissistic personality. To test this, I found a list of personality traits that describe a narcissist (by Googling “children of narcissistic parents” and clicking the first link), and compared with Harry’s personality as described in HPMOR. I got a 100% match. Questions and answers are as follows: 

1. Grandiose sense of self-importance? Check. Harry plans to “optimize” the entire Universe, expects to “do something really revolutionary and important” (Ch. 7), and is trying to “hurry up and become God” (Ch. 27).

2. Obsessed with himself? Check. He appears to only care about people who are smarter or more powerful than him -- people who can help him. He also has contempt for most students and their interests (Quidditch, etc.)

3. Goals are selfish? Check. Harry claims to want to save everyone, but he believes the best way to help others is to increase his own power most quickly. I address two possible objections below:

Harry’s involvement in the Azkaban breakout was selfish, because Harry could not risk losing Quirrell’s friendship: “ It was a bond that went beyond anything of debts owed, or even anything of personal liking, that the two of them were alone in the wizarding world” (Ch. 51). This, again, mirrors a child’s relationship with a narcissistic mother: the child cannot risk losing the mother’s protection. Harry also had selfish reasons for hearing Quirrell’s plan: “There was no advantage to be gained from not hearing it. And if it did reveal something wrong with Professor Quirrell, then it was very much to Harry's advantage to know it, even if he had promised not to tell anyone.” (Ch. 49)

Harry’s efforts to save Hermione are also selfish because Harry sees Hermione in the same way he sees his mother -- weak in many ways and bound by emotions and convention, but someone Harry must impress and protect. Harry’s statement that “it’s disrespectful to her, to think someone could only like her in that way” (ch. 91) makes sense because Harry is disgusted by the Oedipal implications. If Harry’s mother was not narcissistic, then Harry would not have worked so hard to impress Hermione and would have been less disgusted by the thought of being sexually attracted to her.

4. Troubles with normal relationships? Check. Harry is playing high-stakes mind games with the people he is closest to (Quirrell, Draco, Hermione, Dumbeldore), which is not normal friend behavior. Harry has contempt for nearly everyone else.

5. Becomes furious if criticized? Check. When Snape mocked Harry in Potions class, Harry tried to destroy Snape’s career. Quirrell explained, “When it looked like you might lose, you unsheathed your claws, heedless of the danger. You escalated, and then you escalated again” (Ch. 19).

6. Has fantasies of unbound success, power, intelligence, etc.? Check. Harry wants to conquer the entire Universe with the power of his intelligence, and has plans for how to fill an eternity, including to “...meet up with everyone else who was born on Old Earth to watch the Sun finally go out…” (Ch. 39).

7. Believes that he is special and should only be around other high-status people? Check. Harry avoids average students when possible, and certainly does not hang out with them for fun. “Note to self: The 75th percentile of Hogwarts students a.k.a. Ravenclaw House is not the world's most exclusive program for gifted children” (Ch. 12). 

Harry’s association with the (presumably non-special) students in his army is not an exception because minimal text is devoted to Harry instructing them, while much text explains how powerful and high-status the students in the army have become. For Harry, it appears that the army is a tool to use and an opportunity to show off, not an opportunity to give back and help friends improve their skills for their own sake.

8. Requires extreme admiration for everything? Check. Harry takes anything less than admiration for his brilliance as an insult, and responds by striving for new levels of intellectual achievement and arrogance, until the others recognize his dominance. “And I bit a math teacher when she wouldn't accept my dominance” (Ch. 20). Quirrell’s lesson on how to lose described how to avoid making powerful enemies, not how to empathize and care for others -- the insatiable need for admiration is merely delayed and repressed, not corrected.

9. Feels entitled - has unreasonable expectations of special treatment? Check. Harry requires subservience from the school administration, and special magic items such as the time-turner. “McGonagall said, "but I do have a very special something else to give you. I see that I have greatly wronged you in my thoughts, Mr. Potter...this is an item which is ordinarily lent only to children who have already shown themselves to be highly responsible” (Ch. 14).

10. Takes advantage of others to further his own need? Check. Harry justifies his actions toward Draco by saying "I only used you in ways that made you stronger. That's what it means to be used by a friend." (Ch. 97)

11. Does not recognize the feelings of others? Check. One example is Harry not realizing how Neville felt about the prank on the train to Hogwarts. Another is Harry’s remarkably clueless question to Hermione, “Er, can I take it from this that you have been through puberty?" (Ch. 87) Harry has not learned empathy yet: “Harry flinched a little himself. Somewhere along the line he needed to pick up the knack of not phrasing things to hit as hard as he possibly could” (Ch. 86). 

12. Envious or believes they are envied? Check. Quirrell said to Harry, “You have everything now that I wanted then. All that I know of human nature says that I should hate you. And yet I do not. It is a very strange thing.” (Ch. 74)

13. Behaves arrogantly? Check. “Minerva's body swayed with the force of that blow, with the sheer raw lese majeste. Even Severus looked shocked.” (Ch. 19) I can’t think offhand of a single instance when Harry is not arrogant. 

Therefore, I conclude that Harry and Harry’s mother are both narcissistic. If you want further reading on this topic, look up "The Drama of the Gifted Child" by Dr. Alice Miller (Google for the .pdf) for a more detailed description of a child’s typical relationship with a narcissistic parent.

I am sharing this because it reveals a pattern of cognitive biases that many people (like me) who enjoyed HPMOR, and their parents, probably have. Specifically, there is a strong bias toward either narcissistic or people-pleasing habits, and a difficulty with recognizing and following one’s own desires (because the Universe, unlike a parent, never tells people what to do). One possible reason for studying science is to defend against a parent’s emotional neediness and refusal to provide ego-validation by building an impenetrable edifice of logical truth. Unfortunately, identifying the parent’s cognitive biases does not stop their criticism. A more pleasant strategy is to recognize the dynamic, mourn the warping of childhood by the controlling parenting, set appropriate boundaries in the future, and draw validation from following one’s own goals instead of an internalized parent’s goals.

Problems

Problems have bottlenecks. To solve problems, you need to overcome each bottleneck. If you fail to overcome just one bottleneck, the problem will go unsolved, and your effort will have been fruitless.

In reality, it’s a little bit more complicated than that. Some bottlenecks are tighter than others, and some progress might leak through, but it usually isn’t anything notable.

Education

There is a lot wrong with education. Attempts are being made to improve it, but they’re glossing over important bottlenecks. Consequently, progress is slowly dripping through. I think that it’d be a better use of our time to take the time to think through each bottleneck, and how it can be addressed.

I have a theory of how we can overcome enough bottlenecks such that progress will fall through, instead of drip through.

Consider how we learn. Say that you want to learn parent concept A. To do this, it’ll require you to understand a bunch of other things first 

My groundbreaking idea: make sure that students know A₁…A_n before teaching them A.

https://www.dropbox.com/s/4gnwamufalg5gqo/learning.jpg

The bottlenecks to understanding A are A₁…A_n. Some of these bottlenecks are tighter than others, and in reality, there are constraints on our ability to teach, so it’s probably best to focus on the tighter bottlenecks. Regardless, this is the approach we’ll need to take if we want to truly change education.

How would this work?

1) Create a dependency tree.

2) Explain each cell in the tree.

3) Devise a test of understanding for each cell in the tree.

4) Teach accordingly.

Where does our system fail us?

• When you’re in class and the teacher is explaining A when you still don’t get, say A₂ and A₅.
• When you’re in class and the teacher is explaining A, when she never thought to explain A₂ and A₅.
• When you’re reading the textbook and you’re confused, but you don’t even know what child concepts you’re confused about.
• When you memorize for the test/assignment instead of properly filling out your dependency tree.
• When being too far ahead or behind the class leads to a lack of motivation.
• When lack of interest in the material leads to lack of motivation.
• When physical distractions divert your attention (tired, uncomfortable, hungry…).

My proposal

I propose that we pool all of our resources and make a perfect educational web app. It would have the dependency trees, have explanations for each cell in each tree, and have a test of understanding for each cell in each tree. It would test the user to establish what it is that he does and doesn’t know, and would proceed with lessons accordingly.

In other words, usage of this web app would be mastery-based: you’d only proceed to a parent concept when you’ve mastered the child concepts.

Motivation

Motivation would be another thing to optimize.

One way to do this would be to teach things to students at the right times. Lack of interest is often due to lack of understanding of child concepts, and thus lack of appreciation for the beauty and significance of a parent concept. By teaching things to students when they’re able to appreciate them, we could increase students’ motivation. 

Another way to optimize motivation would be to do a better job of teaching students things that are useful to them (or things that are likely to be useful to them). In todays system, students are often times forced to memorize lots of details that are unlikely to ever be useful to them.

By making teaching more effective, I think motivation will naturally increase as well (it’ll eliminate the lack of motivation that comes with the frustration of bad teaching).

Pooling of resources

The pooling of resources to create this web app is analogous to how resources were pooled for Christopher Nolan to make a really cool movie. When you pool resources, a lot more becomes possible. When you don’t pool resources, the product often sucks. Imagine what would happen if you tried to reproduce Batman at a local high school. This is analogous to what we’re trying to do with education now.

How would this look?

I’m not quite sure. Technically, kids could just sit at home on their computers and work through the lessons that the web app gives them… but I sense that that wouldn’t be such a good idea. It’d probably be best to require kids to go to a “school-like institution”. Kids could work through the lessons by themselves, ask each other for help, work together on projects, compete with each other on projects etc.

Certificates

I envision that credentials would be certificate-based. You’d get smaller certificates that indicate that you have mastered a certain subject. Today, the credentials you get are for passing a grade, or passing a class, or getting a degree. They’re too big and inflexible. For example, maybe the plant unit in intro to biology isn’t necessary for you. Smaller certificates allow for more flexibility.

Deadlines

Deadlines are a tough issue. If they exist, there’s a possibility that you have to cram to meet the deadline, and cramming isn’t optimal for learning. However, if they don’t exist, students probably won’t have the incentive to learn. For this reason, I think that they probably do have to exist.

My first thought is that deadlines should be personalized. For example, if I moved 50 steps and the deadline was at 100 steps, the next deadline should be based on where I am now (step 50), not where the deadline was (step 100).

My second thought is that deadlines should be rather loose, because I think that flexibility and personalization are important, and that deadlines sacrifice those things.

My third thought, is that students should be given credit for going faster. In our one-size-fits-all system now, you can’t get credit for moving faster than your class. I think that if you want to work harder and make faster progress, you should be able to and you should be given credentials for the knowledge that you’ve acquired. Given the chance, I think that many students would do this. I think this would allow students to really thrive and pursue their interests.

Tutoring

I think that it’d be a good idea to require tutoring. Say, in order to get a certificate, after passing the tests, you’d have to tutor for x hours.

Tutoring helps you to master the concept, because having to explain something will expose the holes in your understanding. See The Feynman Technique.

Tutoring allows for social interaction, which is important.

Social Atmosphere

The social atmosphere in these “schools” would also be something to optimize. It's not something that people think too much about, but it has a huge impact on how people develop, and thus on how society develops.

I’m not sure exactly what would be best, but I have a few thoughts:

The idea of social value is horrible. In schools today, you grow up caring way too much about how you look, who you’re friends with, how athletic you are, how smart you are, how much success you have with the opposite sex… how “good” you are. This bleeds into our society, and does a lot to cause unhappiness. It should be avoided, if possible.

Relationships are based largely on repeated, unplanned interactions + an environment that encourages you to let your guard down. I think that schools should actively provide these situations to students, and should allow you to experience these situations with a variety of types of people (right now you only get these repeated, unplanned interactions with the cohort of students you happen to be with, which limits  you in a lot of ways).

Rationality

I propose that rationality be a core part of the curriculum (the benefits of making people better at reasoning would trickle down into many aspects of life). I think that this should be done in two ways: the first is by teaching the ideas of rationality, and the second is by using them.

The ideas of rationality can be found right here. Some examples:

• Your beliefs have to be about anticipated experiences.
• Don’t commit the fallacy of gray.
• Understanding that you should optimize the terminal value.
• Don’t treat arguments like war.
• Disagree by refuting the central point.
• Be specific.

After the ideas are taught, they should be practiced. The best way that I could think of to do this is to have kids write and critique essays (writing is just thought on paper, and it’s often easier to argue in writing than it is in verbal conversation). Students could pick a topic that they want to talk about, make claims, and argue for them. And then they could read each others’ essays, and point out what they think are mistakes in each others’ reasoning (this should all be supervised by a teacher, who should probably be more of a benevolent dictator, and who should also contribute points to the discussions).

I think that some competition and social pressure could be useful too; maybe it’d be a good idea to divide students into classes, where the most insightful points are voted upon, and the number of mistakes committed would be tallied and posted.

Writing

Right now, essays in schools are a joke. No one takes them seriously. Students b.s. them, and teachers barely read them, and hardly give any feedback. And they’re also always on english literature, which sends a bad message to kids about what an essay really is. Good writing isn’t taught or practiced, and it should be.

Levels of Action

Certain levels of action have impacts that are orders of magnitude bigger than others. I think that improving education this much would be a high level action, and have many positive effects that’ll trickle down into many aspects of society. I’ll let you speculate on what they are.

Does anyone know of a good article that illustrates how society is generally irrational, and how making society more rational would have huge benefits, because it'd be a very high level action?

I'm writing an essay about how to improve education, and one of my proposals is that a core part of the curriculum should be rationality. I believe that doing this would have huge benefits to society, and want to explain why I think this, but I'm having trouble. Any thoughts?

Edit: Part of Raising the Sanity Waterline talks about common ways in which people are irrational. However, they're all links to longer Less Wrong articles. Preferably, I'd like to illustrate it in a few sentences/paragraphs.

So I have a three-year old kid, and will usually read or tell him a bedtime story.

That is a nice opportunity to introduce new concepts, but my capacity for improvisation is limited, especially towards the end of the day. So I'm asking the good people on LessWrong for ideas. How would you wrap various lesswrongish ideas in a short story a little kid would pay attention to?

I'm mostly interested in the aspects of "practical rationality" that aren't going to be taught at school or in children's books or children's TV shows - so things like Sunk Costs, taking the outside view, wondering which side is true instead of arguing for a side, etc.

Pointers to outside sources of such stories are welcome too!

Edit: actually, if you want to share ideas of games or activities of the same kind, go ahead! :)

I wrote this post up and circulated it among my rationalist friends. I've copied it verbatim. I figure the more rationally inclined people that can critique my plan the better.

--

TL;DR:

* I'm going to commit to biomedical engineering for a very specific set of reasons related to career flexibility and intrinsic interest.
* I still want to have computer science and design arts skills, but biomedical engineering seems like a better university investment.
* I would like to have my cake and eat it too by doing biomedical engineering, while practicing computer science and design on the side.
* There are potential tradeoffs, weaknesses and assumptions in this decision that are relevant and possibly critical. This includes time management, ease of learning, development of problem solving solving abilities and working conditions.

I am posting this here because everyone is pretty clever and likes decisions. I am looking for feedback on my reasoning and the facts in my assumptions so that I can do what's best. This was me mostly thinking out loud, and given the timeframe I'm on I couldn't learn and apply any real formal method other than just thinking it through. So it's long, but I hope that everyone can benefit by me putting this here.

--
So currently I'm weighing going into biomedical engineering as my major over a major in computer science, or the [human-computer interaction/media studies/gaming/ industrial design grab bag] major, at Simon Fraser University. Other than the fact that engineering biology is so damn cool, the relevant decision factors include reasons like:

1. medical science is booming with opportunities at all levels in the system, meaning that there might be a lot of financial opportunity in more exploratory economies like in SV;
2. the interdisciplinary nature of biomedical engineering means that I have skills with greater transferability as well as insight into a wide range of technologies and processes instead of a narrow few;
3. aside from molecular biology, biomedical engineering is the field that appears closest to cognitive enhancement and making cyborgs for a living;
4. compared to most kinds of engineering, it is more easy to self-teach computer science and other forms of digital value-making (web design or graphical modelling) due to the availability of educational resources; the approaching-free cost of computing power; established communities based around development; and clear measurements of feedback. By contrast, biomedical engineering may require labs to be educated on biological principles, which are increasingly available but scarce for hobbyists; basic science textbooks are strongly variant in quality; and there isn't the equivalent of a Github for biology making non-school collaborative learning difficult.

The two implications here are that even if I am still interested in computer science, which I am, and although biomedical engineering is less upwind than programming and math, it makes more sense to blow a lot of money on a more specialized education to get domain knowledge while doing computer science on the side, than to spend money on an option whose potential cost is so low because of self study. This conjecture, and the assumptions therein, is critical to my strategy.

So the best option combination that I figure that I should take is this:

1. To get the value from Biomedical Engineering, I will do the biomedical engineering curriculum formally at SFU for the rest of my time there as my main focus.
2. To get the value from computer science, I will make like a hacker and educate myself with available textbooks and look for working gigs in my spare time.
3. To get the value from the media and design major, I will talk to the faculty directly about what I can do to take their courses on human computer interaction and industrial design, and otherwise be mentored. As a result I could seize all the real interesting knowledge while ignoring the crap.

Tradeoffs exist, of course. These are a few that I can think of:

• I don't expect to be making as much as an entry level biomedical engineer as I would as a programmer in Silicon Valley, if that was ever possible; nor do I believe that my income would grow at the same rate. As a counterpoint, my range of potential competencies will be greater than the typical programmer, due to an exposure to physical, chemical, and biological systems, their experimentation, and product development. I feel that this greater flexibility could help with companies or startups that are oriented towards health or technological forecasting, but this is just a guess. In any case that makes me feel more comfortable, having that broader knowledge, but one could argue that programming being so popular and upwind makes it the more stable choice anyway. Don't know.
• It's difficult to make money as an undergraduate with any of the skills I would pick up in biomedical engineering for at least a few years. This is important to me because I want to have more-than-minimum wages jobs as a way of completing my education on a debit. While web and graphic designers can start forming their own employment almost immediately, and while programmers can walk into a business or a bank and hustle; doing so with physics, chemistry or biology seems a bit more difficult. This is somewhat countered by co-op and work placement, and the fact that it doesn't seem to take too much programming or web design theory and practice before being able to start selling your skills (i.e. on the order of months).
• Biomedical Engineering has few aesthetic and artistic aspects, the two of which I value. This is what attracted me to the media and design program in the first place. Instead I get to work with technologies which I know will have measurable and practical use, improving the quality of life for the sick and dying. Expressing myself with art and more free-wheeling design is not super urgent, so I'm willing to make this trade. I still hope to be able to orient myself for developing beautiful and useful data visualizations in practical applications, like this guy, and to experiment with maker hacking.

There is still the issue of assuring more-than-dilettante expertise in computer science and design stuff (see Expert Beginner syndrome: http://www.daedtech.com/how-developers-stop-learning-rise-of-the-expert-beginner). I am semi-confident in my ability to network myself into mentorships with members of faculty [at SFU] that are not my own, and if I'm not good at it now I still believe that it's possible. In addition, my dad has recently become a software consultant and is willing to apprentice me, giving a direct education about software engineering (although not necessarily a good one, at least it's somewhat real).

There are potential weaknesses in my analysis and strategy.

• The time investment in the biomedical engineering faculty as SFU is very high. The requirements are similar to those of being a grad student, complete with a 3.00 minimum GPA and research project. The faculty does everything in its power to allay the burden while still maintaining the standard. However, this crowding out of time reduces the amount of potential time spent learning computer science. This makes the probability of efficient self-teaching go down. (that GPA standard might lead to scholarship access which is good, but more of an externality in this case.)
• While we're on the conscientiousness load: conscientiousness is considered to be an invariant personality trait, but I'm not buying it. The typical person may experience on average no change in their conscientiousness, but typical people don't commit to interventions that affect the workload they can take on either by strengthening willpower, increasing energy, changing thought patterns (see "The Motivation Hacker") or improving organization through external aids. Still, my baseline level of conscientiousness has historically been quite low. This raises the up front cost of learning novel material I'm not familiar with, unlike computing, of which I have a stronger familiarity due to lifelong exposure; this lets me cruise by in computing courses but not necessarily ace them. Nevertheless, that's a lower downside risk.
• Although medical problems are interesting and I have a lot of intrinsic interest in the domain knowledge, there are components of research that interest me while others that I don't currently enjoy as much as evidenced from my current exposure. I can seem myself getting into the data processing and visualization, drafting ergonomic wearable tech, and circuit design especially wrt EEGs. Brute force labwork would be less engaging and takes more out of me, despite systems biology principles being tough but engaging. So there's the possibility that I would only enjoy a limited scope of biomedical engineering work, making the major not worth it or unpleasant.
• Due to the less steep learning curve and more coherent structure of the computer science field, it seems easier to approach the "career satisfaction" or "work passion" threshold with CS than for BME. Feeling satisfied with your career depends on many factors, but Cal Newport argues that the largest factor is essentially mastery, which leads to involvement. Mastery seems more difficult to guage with the noisy and prolonged feedback of the engineering sciences, so the motivations with the greatest relative importance might be the satisfaction of turning out product, satisfying factual curiosity or curiosity about established/canon models (as opposed to curiosity which is more local to your own circumstances or you figuring things out), and in the case of biomed, saving lives by design. With mathematics and programming the problem space is such that you can do math and programming for their own sakes.
• Most instances of biomedical engineering majors around the world are mainly graduate studies. The most often reported experience is that when you have someone getting a PhD in biomedical engineering, it's in addition to their undergraduate experience as a mechanical engineer, an electrical engineer or a computer scientist. The story goes that these problem solving skills are applied to the biology after being developed - once again a case of some fields being more upwind than others. By contrast, an undergradute in bioengineering would be taking courses where they are not developing these skills, as our current understanding of biology is not strongly predictive. After talking to one of the faculty heads, the person who designed the program, he is very much aware of problems such as these in engineers as they are currently educated. This includes overdoing specialization and under-emphasizing the entire product development process, or a principle of "first, do no harm". He has been working on the curriculum for thirty years as opposed to the seven years of cases like MIT - I consider this moderate evidence that I will not be missing out on the necessary mental toolkit over other engineers.
• In the case where biomedical engineering is less flexible than I believed, I would essentially have a "jack of all trades" education meaning engineering firms in general would pass over me in favor of a more specialized candidate. This is partially hedged against by learning the computer science as an "out", but in the end it points to the possibility that the way I'm perceiving this major's value is incorrect.

So for this "have cake and eat it to" plan to work there are a larger string of case exceptions in the biomedical option than the computing options, and definitely the media and design option. The reward would be that the larger amount of domain specific knowledge in a field that has held my curiosity for several years now, while hitting on. I would also be playing to one of SFU's comparative advantages: the quality of the biomedical faculty here is high relative to other institutions if the exceptions hold, and potentially the relative quality of the computer science and design faculties as well. (This could be an argument for switching institutions if those two skillsets are a "better fit". However, my intuition is that the cost for such is very high and probably wouldn't be worth it.)

Possible points of investigation:

• What is hooking me most strongly to biomedical engineering were the potentials of cognitive enhancement research and molecular design (like what they have going on at the bio-nano group at Autodesk: http://www.autodeskresearch.com/groups/nano). If these were the careers I was optimizing towards as an ends, it might make more sense to actual model what skills and people will actually be needed to develop these technologies and take advantage of them. After writing this I feel less strongly about these exact fields or careers. Industry research still seems like a good exercise.
• I will have to be honest that after my experience doing lab work for chemistry at school, I was frustrated by how exhausted I am at the end of each session, physically and mentally. This doesn't necessarily reflect on how all lab work will be, especially if it's more intimately tied with something else I want to achieve. And granted, the labs are three hours long of standing. It does make me question how I would be like in this work environment, however, and that is worth collecting more information for.
• To get actual evidence of flexibility in skillset it would be worth polling actual alumni from the program, to see if any of the convictions about the program are true.

--

Thoughts, anyone?

I'm working on a post about how best to use human memory—when it's good to store things in your own brain and why, when it's best to outsource your memory, what memory upgrades are worthwhile in what contexts, and how to integrate and apply memory systems in real life. I'm hoping the following set of memory problems will draw out approaches that haven't occurred to me so I can compare a wider range of methods.

I'll post the first solutions I thought of myself later on, but for now I'd like to hear what you would do in each of these situations and what you believe to be the pros and cons of your answers. Can you think of ways to improve upon your first thoughts and the answers of others?

(You don't have to respond to all of the questions; feel free to post as little or as much as comes to mind.)

I'm finally getting around to reading "Thinking, Fast and Slow". Much of it I had already learned on LW and elsewhere. Maybe that's why my strongest impression from the book is how accessible it is. Simple sentences, clear and vivid examples, easy-to-follow exercises, a remarkable lack of references to topics not explained right away.

I caught myself thinking "This is a book I should have read as a kid". In my first language, I think I could have managed it as early as 11 years old. Since measured IQ is strongly influenced by habits of thinking and cognitive returns can be reinvested, I'm sure I would be smarter now if I had.

So I have decided to buy a stack of these books and give them to kids on their, say, 12th birthdays. Then maybe Dan Dennett's "Intuition Pumps" a year later - and HPMOR a year after that? I would like to see more suggestions from you guys.

It should be obviously better to start even earlier. So how do you teach rationality to a nine-year-old? Or a seven-year-old? Has anybody done something like that? Please name books, videos or web sites.

If such media are not available, creating them should be low-hanging fruit in the quest to raise the global IQ and sanity waterline. ELI5 writing is very learnable, after all, and ELI5 type interpretations of, say, the sequences, might be helpful for adults too.

I'm reading Nurture Shock by Po Bronson & Ashley Merryman. Several things in the book, esp. the chapter on "Tools of the Mind", an intriguing education program, suggest that our education of young children not only isn't very good even when evaluated using tests that the curriculum was designed for, it's worse than just letting kids play. (My analogy and interpretation—don't blame this on the Tools people—is that conventional education may be like a Soviet five-year plan, trying to force children to acquire skills & knowledge that they would have been motivated to learn on their own if there weren't a school, and that early education shouldn't focus entirely on teaching specific facts, but also on teaching how to think, form abstractions, and control impulses.)

Say they're going to play fireman. The Tools teacher teaches the kids about what firemen do and what happens in a fire, and gives the kids different roles to play, then lets them play. They teach facts not because the facts are important, but to make the play session longer and more complicated.  Tools does well in increasing test scores, but even better at reducing disruptive behavior. [1]

Tools has a variety of computer games that are designed to get kids to exercise particular cognitive skills, like focusing on something while being aware of background events. But the games often sound like more-boring ways of teaching kids the same things that video-games teach them.

Tools did no better than the existing curriculum on certain metrics in a recent larger study. But it didn't perform worse, either.

The first study you do with any biological intervention is to compare the intervention to a control group that has no intervention. But in education, AFAIK no one has ever done this. Everyone uses the existing curriculum as the control.

Whatever country you're in, what metrics do you use, and what evidence do you have that your schools are better than nothing at all?

There may be some things that you need to sit kids down and force them to learn—say, arithmetic, math, and typing—but I kinda doubt it's more than 20% of the grade school curriculum. I spent a lot of time practicing penmanship, futilely trying to memorizing the capitals and chief exports of all fifty states, and studying the history of Thanksgiving and the American Revolution over and over again.[2] We could have a short-hours classroom hours control group, where kids spend a few hours a day learning those few facts they need to know, and the rest of the time playing.

ADDED: There is one kind of control--kids who've not gone to pre-school vs. kids who went to pre-school, or who went to Head Start.

[1] I fear somebody is going to complain that disruptive behavior is what we need to teach children so they can innovate and question authority. Open to discussion, but if it worked that way, we'd be overwhelmed with innovators and independent thinkers today.

[2] I actually learned the names of all the states from a song, and learned where they are from a jigsaw puzzle.

Preamble

Before I make my main point, I want to acknowledge that curriculum development is hard. It's even harder when you're trying to teach the unteachable. And it's even harder when you're in the process of bootstrapping. I am aware of the Kahneman inside/outside curriculum design story. And, I myself have taught 200+ hours of my own computer science curricula to middle-school students. So this "open letter," is not some sort of criticism of CFAR's curriculum; It's a "Hey, check out this cool stuff eventually when you have time," letter. I just wanted to put all this out there, to possibly influence the next five years of CFAR.

Curriculum development is hard.

So, anyway, I don't personally know any of the people involved in CFAR, but I do know you're all great. 

A case for developmental thinking

Below is an annotated bibliography of some of my personal touchstones in the development literature, books that are foundational or books that synthesize decades of research about the developmental aspects of entrepreneurial, executive, educational, and scientific thinking, as well as the developmental aspects of emotion and cognition. Note that this is personal, idiosyncratic, non-exhaustive list.

And, to qualify, I have epistemological and ontological issues with plenty of the stuff below. But some of these authors are brilliant, and the rest are smart, meticulous, and values-driven. Lots of these authors deeply care about empirically identifying, targeting, accelerating, and stabilizing skills ahead of schedule or helping skills manifest when they wouldn't have otherwise appeared at all. Quibbles and double-takes aside, there is lots of signal, here, even if it's not seated in a modern framework (which would of course increase the value and accessibility of what's below).

There are clues or even neon signs, here, for isolating fine-grained, trainable stuff to be incorporated into curricula. Even if an intervention was designed for kids, a lot of adults still won't perform consistently prior to said intervention. And these researchers have spent thousands of collective hours thinking about how to structure assessments, interventions, and validations which may be extendable to more advanced scenarios.

So all the material below is not only useful for thinking about remedial or grade-school situations, and is not just for adding more tools to a cognitive toolbox, but could be useful for radically transforming a person's thinking style at a deep level.

Consider:

child:adult :: adult: ? 

This has everything to do with the "Outside the Box" Box. Really. One author below has been collecting data for decades to attempt to describe individuals that may represent far less than one percent of the population.

0. Protocol analysis

Everyone knows that people are poor reporters of what goes on in their heads. But this is a straw. A tremendous amount of research has gone into understanding what conditions, tasks, types of cognitive routines, and types of cognitive objects foster reliable introspective reporting. Introspective reporting can be reliable and useful. Grandaddy Herbert Simon (who coined the term "bounded rationality") devotes an entire book to it. The preface (I think) is a great overview. I wanted to mention this, first, because lots of the researchers below use verbal reports in their work.

http://www.amazon.com/Protocol-Analysis-Edition-Verbal-Reports/dp/0262550237/

1. Developmental aspects of scientific thinking

Deanna Kuhn and colleagues develop and test fine-grained interventions to promote transfer of various aspects of causal inquiry and reasoning in middle school students. In her words, she wants to "[develop] students' meta-level awareness and management of their intellectual processes." Kuhn believes that inquiry and argumentation skills, carefully defined and empirically backed, should be emphasized over specific content in public education. That sounds like vague and fluffy marketing-speak, but if you drill down to the specifics of what she's doing, her work is anything but. (That goes for all of these 50,000 foot summaries. These people are awesome.)

http://www.amazon.com/Education-Thinking-Deanna-Kuhn/dp/0674027450/

http://www.tc.columbia.edu/academics/index.htm?facid=dk100

http://www.educationforthinking.org/

David Klahr and colleagues emphasize how children and adults compare in coordinated searches of a hypothesis space and experiment space. He believes that scientific thinking is not different in kind than everyday thinking. Klahr gives an integrated account of all the current approaches to studying scientific thinking. Herbert Simon was Klahr's dissertation advisor.

http://www.amazon.com/Exploring-Science-Cognition-Development-Discovery/dp/0262611767

http://www.psy.cmu.edu/~klahr/

2. Developmental aspects of executive or instrumental thinking

Ok, I'll say it: Elliot Jacques was a psychoanalyst, among other things. And the guy makes weird analogies between thinking styles and truth tables. But his methods are rigorous. He has found possible discontinuities in how adults process information in order to achieve goals and how these differences relate to an individuals "time horizon," or maximum time length over which an individual can comfortably execute a goal. Additionally, he has explored how these factors predictably change over a lifespan.

http://www.amazon.com/Human-Capability-Individual-Potential-Application/dp/0962107077/

3. Developmental aspects of entrepreneurial thinking

Saras Sarasvathy and colleagues study the difference between novice entrepreneurs and expert entrepreneurs. Sarasvathy wants to know how people function under conditions of goal ambiguity ("We don't know the exact form of what we want"), environmental isotropy ("The levers to affect the world, in our concrete situation, are non-obvious"), and enaction ("When we act we change the world"). Herbert Simon was her advisor. Her thinking predates and goes beyond the lean startup movement.

http://www.amazon.com/Effectuation-Elements-Entrepreneurial-Expertise-Entrepreneurship/dp/1848445725/

"What effectuation is not" http://www.effectuation.org/sites/default/files/research_papers/not-effectuation.pdf

Related: http://lesswrong.com/r/discussion/lw/hcb/book_suggestion_diaminds_is_worth_reading/

4. General Cognitive Development

Jane Loevinger and colleagues' work have inspired scores of studies. Loevinger discovered potentially stepwise changes in "ego level" over a lifespan. Ego level is an archaic-sounding term that might be defined as one's ontological, epistemological, and metacognitive stance towards self and world. Loevinger's methods are rigorous, with good inter-rater reliability, bayesian scoring rules incorporating base rates, and so forth.

http://www.amazon.com/Measuring-Ego-Development-Volume-Construction/dp/0875890598/

http://www.amazon.com/Measuring-Development-Scoring-Manual-Women/dp/0875890695/

Here is a woo-woo description of the ego levels, but note that these descriptions are based on decades of experience and have a repeatedly validated empirical core. The author of this document, Susanne Cook-Greuter, received her doctorate from Harvard by extending Loevinger's model, and it's well worth reading all the way through: 

http://www.cook-greuter.com/9%20levels%20of%20increasing%20embrace%20update%201%2007.pdf

Here is a recent look at the field:

http://www.amazon.com/The-Postconventional-Personality-Researching-Transpersonal/dp/1438434642/

By the way, having explicit cognitive goals predicts an increase in ego level, three years later, but not an increase in subjective well-being. (Only the highest ego levels are discontinuously associated with increased wellbeing.) Socio-emotional goals do predict an increase in subjective well-being, three years later. Great study:

Bauer, Jack J., and Dan P. McAdams. "Eudaimonic growth: Narrative growth goals predict increases in ego development and subjective well-being 3 years later." Developmental Psychology 46.4 (2010): 761.

5. Bridging symbolic and non-symbolic cognition

[Related: http://wiki.lesswrong.com/wiki/A_Human's_Guide_to_Words]

Eugene Gendlin and colleagues developed a "[...] theory of personality change [...] which involved a fundamental shift from looking at content [to] process [...]. From examining hundreds of transcripts and hours of taped psychotherapy interviews, Gendlin and Zimring formulated the Experiencing Level variable. [...]"

The "focusing" technique was designed as a trainable intervention to influence an individual's Experiencing Level.

Marion N. Hendricks reviews 89 studies, concluding that [I quote]:

• Clients who process in a High Experiencing manner or focus do better in therapy according to client, therapist and objective outcome measures.
• Clients and therapists judge sessions in which focusing takes place as more successful.
• Successful short term therapy clients focus in every session.
• Some clients focus immediately in therapy; Others require training.
• Clients who process in a Low Experiencing manner can be taught to focus and increase in Experiencing manner, either in therapy or in a separate training.
• Therapist responses deepen or flatten client Experiencing. Therapists who focus effectively help their clients do so.
• Successful training in focusing is best maintained by those clients who are the strongest focusers during training.

http://www.focusing.org/research_basis.html

http://www.amazon.com/Focusing-Eugene-T-Gendlin/dp/0553278339/

http://www.amazon.com/Focusing-Oriented-Psychotherapy-Manual-Experiential-Method/dp/157230376X/

http://www.amazon.com/Self-Therapy-Step-By-Step-Wholeness-Cutting-Edge-Psychotherapy/dp/0984392777/ [IFS is very similar to focusing]

http://www.amazon.com/Emotion-Focused-Therapy-Coaching-Clients-Feelings/dp/1557988811/ [more references, similar to focusing]

http://www.amazon.com/Experiencing-Creation-Meaning-Philosophical-Psychological/dp/0810114275/ [favorite book of all time, by the way]

6. Rigorous Instructional Design

Siegfried Engelmann (http://www.zigsite.com/) and colleagues are dedicated to dramatically accelerating cognitive skill acquisition in disadvantaged children. In addition to his peer-reviewed research, he specializes in unambiguously decomposing cognitive learning tasks and designing curricula. Engelmann's methods were validated as part of Project Follow Through, the "largest and most expensive experiment in education funded by the U.S. federal government that has ever been conducted," according to Wikipedia. Engelmann contends that the data show that Direct Instruction outperformed all other methods:

http://www.zigsite.com/prologue_NeedyKids_chapter_5.html

http://en.wikipedia.org/wiki/Project_Follow_Through

Here, he systematically eviscerates an example of educational material that doesn't meet his standards:

http://www.zigsite.com/RubricPro.htm

And this is his instructional design philosophy:

http://www.amazon.com/Theory-Instruction-Applications-Siegfried-Engelmann/dp/1880183803/

Conclusion

In conclusion, lots of scientists have cared for decades about describing the cognitive differences between children, adults, and expert or developmentally advanced adults. And lots of scientists care about making those differences happen ahead of schedule or happen when they wouldn't have otherwise happened at all. This is a valuable and complementary perspective to what seems to be CFAR's current approach. I hope CFAR will eventually consider digging into this line of thinking, though maybe they're already on top of it or up to something even better.

David McClelland published an influential article (1973) claiming that IQ tests have no value, because they do not correlate with success and it is not clear that they measure anything other than social status. McClelland opened up a new discussion of whether tests predict career success, and whether the purpose of education is social investment or social reformation (why would we even want to single out children with high IQs if those are the children we want not to educate, in order to level the playing field?)

This work is controversial, maybe even more so today than in the 1980s. (Barrett & Depinet 1991) accused McClelland of simply lying, by not mentioning most studies that disagreed with his conclusions and misrepresenting the results of those he did quote.

But in all this time, no one has asked the most-important question: Should we try to make (other people's) children more successful? And should we deliberately promote children because they're likely to be successful?

(If the answer is yes, perhaps we should focus on giving more opportunities to children of the wealthy, since parental wealth is the strongest correlate with career success.)

A close look at (Barrett & Depinet 1991) suggests that, when social class is factored out, IQ correlates well with objective measures of performance, such as employee evaluations, ratings of work samples, and production quantity, but poorly with measures of career success such as job title and salary. Social intelligence is thus the stuff that improves your career but not your performance. That sounds suspiciously like it's skills that help you put one over on your co-workers.

Success is a zero-sum game. It's measured by your position and wealth relative to other people. It makes sense for a prep school or college to advertise that they will make you more successful. It doesn't make sense for a taxpayer-funded school system to do so. Public school is funded by the public in order to benefit the public. The public wants performance, not career success, from you.

It's no paradox that IQ correlates more with performance than with success. Social intelligence does wonders for your career success. People with high social intelligence are able to drive their (often stupid) ideas through committees by using coalition-building and hate-mongering, as well as sarcasm, dismissive humor, emotionally-laden jargon ("death tax"), distraction, and a fine sense of when they can use argument by assumption. They are the people who get grants by schmoozing, playing off the prejudices of the review panel, and snappy data-free PowerPoint presentations. They are the artists who paint a canvas black and then publish a three-page explanation of how that is a criticism of art consumerism. They are good at getting raises, bonuses, and promotions, and at taking credit for other people's work. They are the people who are ruining science and art.

Think that a boss with high social intelligence will make your work more pleasant and resolve conflicts with your co-workers? Maybe. Or maybe that boss will strategically create conflicts to foster competition, and use their superior social intelligence to make you work harder and longer for less pay.

(There is an underlying assumption behind how all this testing is applied that the same skills make a person a good worker and a good manager. I'm not even going to touch that question, especially since behind it lies the even harder question, "A manager good for whom, the company or the worker?")

It can make sense to teach social skills to people who lack them, but it doesn't make a lot of sense to fast-track people for having competitive skills at zero-sum contests. Teaching everyone skills that would maximize their individual competitiveness if no one else has those skills may have no net effect. Putting people into gifted programs or admitting them into more-elite colleges because they have high social skills might mean that people with higher intelligence (and better ideas) will have a harder time getting their views heard. Give me a workplace full of stuttering nerds with pocket protectors, not conniving manipulators.

Social skills may be an important and overlooked part of education. But we shouldn't uncritically overhaul our educational system without looking carefully at what we're maximizing for.


References

Gerald Barrett, Robert Depinet (1991). A reconsideration of testing for competence rather than for intelligence. American Psychologist 46(10), Oct 1991, 1012-1024.

David C. McClelland (1973). Testing for competence rather than for "intelligence". American Psychologist 28(1), Jan 1973, 1-14. doi: 10.1037/h0034092.

David Payne, Patrick Kyllonen (2012). The role of noncognitive skills in academic success. Presented at 21st Century knowledge and skills: the new curriculum and the future of assessment. Los Angeles, California, January 11-13, 2012.

I graduated from high school and wish to further my education formally by studying for a bachelor's degree in order to become a medical researcher. I could, for instance, take two different academic paths:

1. Study Medicine at undergraduate level and then do a postdoctoral fellowship.

2. Study Biochemistry at undergraduate level, then study for a PhD at graduate level, and finally do a postdoctoral fellowship.

Since I will do these studies in Europe, they each take approximately the same amount of time, namely 6 to 8 years.

Do I want to do treat patients? No, I do not. But I am considering Medicine because it can be a buffer against my own mediocrity: in case I turn out to be a below average scientist, I will be screwed royally. From my personal job shadowing experience, Medicine, on the other hand, requires mere basic intellectual traits, primarily the ability to memorize heaps of information. And those I think I have. To do world-class research though I'd have to be an intellectual heavyweight, and of that I'm not so sure.

How do I decide what path to  follow?

The reason I'm asking you strangers for advice is because I evidently have biases, such as the pessimism/optimism bias or the Dunning–Kruger effect, that impair my ability to reason clearly; and people who know me personally are likewise prone to make errors in advising me because of biases like, say, the Halo effect. (Come to think of it, thinking that I can't become an above average scientist is in itself a self-defeating prophecy!)

Do you think that one ought to always seek advice from total strangers in order to be safeguarded from his/her own biases?

PS: I apologize if I should have written this in a specific thread. I'll delete my article if that's necessary.

As a teacher, I wonder if it is possible to instill this skill into students the skills of rationality and critical thinking. I teach the third grade, and it is not immediately apparent how to apply this with my own class.

The problems I foresee are as follows:

• Young children often do not know the basics on the subject which they are learning, be it math, science, art, religion, literature etc.
• Many children are very shy, and try to give as short of an answer as doable to a verbal prompt.
• Written prompts are arduous, straining the attention span and writing capabilities of the students. This is not a bad thing, but it presents difficulties in the economy of time and material to be presented.
• Attention spans in general are very short.
• Experiments can be very infrequent, and nigh impossible with certain subjects.
• Children, at this age, are likely to take the words of a parent or teacher at face value, and naturally parrot it back. This may be a hard habit to break.

In the sequences, it is suggested teachers should drill into students words don't count, only anticipation-controllers. How practical is this for an elementary school level? Also appreciated would be any ideas or experiences on how to do this, or how to combat the above problems. Hearing from other teachers would be excellent especially.

I wish to transfer to a university in Europe, to complete my engineering formation. I thought it might be the opportunity to initiate a discussion on the merits of European technical schools, given how many people here have a STEM background, and have experienced the first-hand.

Which ones do you think are best at teaching? Which provide the best starting point, professionally? Which have the most productive, idealistic mood among the studentship? If you've been to several of schools, how do they compare to each other?

The floor is yours.

Coursera has an upcoming course on critical thinking called "Think Again: How to Reason and Argue". A bunch of the locals here in Columbus will be taking it, and I thought some other LWers might also be interested. If you decide to sign up, let us know, and we might get a discussion group going, or something.

Linky: https://www.coursera.org/course/thinkagain 

Some info from the website:

About the Course: 

Reasoning is important.  This course will teach you how to do it well.  You will learn some simple but vital rules to follow in thinking about any topic at all and some common and tempting mistakes to avoid in reasoning.  We will discuss how to identify, analyze, and evaluate arguments by other people (including politicians, used car salesmen, and teachers) and how to construct arguments of your own in order to help you decide what to believe or what to do. These skills will be useful in dealing with whatever matters most to you.

Course Syllabus

Week One: How to Spot an Argument (and separate it from surrounding verbiage)

Week Two: How to Untangle an Argument (or break it into parts and tell what different parts are doing)

Week Three: How to Reconstruct an Argument (or arrange its parts to show how they are connected in a structure) 

Week Four: How to Evaluate an Argument Deductively (or determine whether its conclusion follows validly from its premises) – Part 1: Propositional Logic

Week Five:  How to Evaluate an Argument Deductively (or determine whether its conclusion follows validly from its premises) – Part 2: Quantificational Logic

Week Six: How to Evaluate an Argument Inductively (or assess whether its premises provide enough reason to believe its conclusion) – Part 1: Statistical Generalization and Application

Week Seven: How to Evaluate an Argument Inductively (or assess whether its premises provide enough reason to believe its conclusion) – Part 2: Causal Reasoning

Week Eight:  How to Evaluate an Argument Inductively (or assess whether its premises provide enough reason to believe its conclusion) – Part 3: Probability 

Week Nine:  How to Evaluate an Argument Inductively (or assess whether its premises provide enough reason to believe its conclusion) – Part 4: Decisions

Week Ten: How to Mess Up an Argument (or commit common but tempting fallacies) – Part 1: Vagueness and Ambiguity

Week Eleven: How to Mess Up an Argument (or commit common but tempting fallacies) – Part 2: Irrelevance

Week Twelve: How to Mess Up an Argument (or commit common but tempting fallacies) – Part 3: Vacuity

Some time ago I learned of the metaphor of 'digging the bull's horn'. This might sound a little strange, since horns are mostly hollow, but imagine a bull's horn used to store black powder. In the beginning the work is easy and you can scoop out a lot powder with very little effort. As you dig down, though, each scoop yields less powder as you dig into the narrow part of the horn until the only way you can get out more powder is to turn the horn over a dump it out.

It's often the same way with learning. When you start out in a subject there is a lot to be learned (both in quantity of material you have not yet seen and in quantity of benefits you have to gain from the information), but as you dig deeper into a subject the useful insights come less often or are more limited in scope. Eventually you dig down so far that the only way to learn more is to discover new things that no one has yet learned (to stretch the metaphor, you have to add your own powder back to dig out).

It's useful to know that you're digging the bull's horn when learning because, unless you really enjoy a subject or have some reason to believe that contributing to it is worthwhile, you can know in advance that most of the really valuable insights you'll gain will come early on. If you want to benefit from knowing about as much stuff as possible, you'll often want to stop actively pursuing a subject unless you want to make a career out of it.

But, for a few subjects, this isn't true. Sometimes, as you continue to learn the last few hard things that don't seem to provide big, broadly-useful insights, you manage to accumulate a critical level of knowledge about the subject that opens up a whole new world of insights to you that were previously hidden. To push the metaphor, you eventually dig so deep that you come out the other side to find a huge pile of powder.

The Way seems to be one of those subjects you can dig past the end of: there are some people who have mastered The Way to such an extent that they have access to a huge range of benefits not available to those still digging the horn. But when it comes to other subjects, how do you know? Great insights could be hiding beyond currently obscure fields of study because no one has bothered to dig deep enough. Aside from having clear examples of people who came out the other side to give us reason to believe it's worth while to deep really deep on some subjects, is there any way we can make a good prediction about what subjects may be worth digging to the end of the bull's horn?

Just read this article, which describes a splashy, interesting narrative which jives nicely with my worldview. Which makes me suspicious.

http://dvice.com/archives/2012/10/ethiopian-kids.php

The One Laptop Per Child project started as a way of delivering technology and resources to schools in countries with little or no education infrastructure, using inexpensive computers to improve traditional curricula. What the OLPC Project has realized over the last five or six years, though, is that teaching kids stuff is really not that valuable. Yes, knowing all your state capitols how to spell "neighborhood" properly and whatnot isn't a bad thing, but memorizing facts and procedures isn't going to inspire kids to go out and learn by teaching themselves, which is the key to a good education. Instead, OLPC is trying to figure out a way to teach kids to learn, which is what this experiment is all about.

Rather than give out laptops (they're actually Motorola Zoom tablets plus solar chargers running custom software) to kids in schools with teachers, the OLPC Project decided to try something completely different: it delivered some boxes of tablets to two villages in Ethiopia, taped shut, with no instructions whatsoever. Just like, "hey kids, here's this box, you can open it if you want, see ya!"

Just to give you a sense of what these villages in Ethiopia are like, the kids (and most of the adults) there have never seen a word. No books, no newspapers, no street signs, no labels on packaged foods or goods. Nothing. And these villages aren't unique in that respect; there are many of them in Africa where the literacy rate is close to zero. So you might think that if you're going to give out fancy tablet computers, it would be helpful to have someone along to show these people how to use them, right?

But that's not what OLPC did. They just left the boxes there, sealed up, containing one tablet for every kid in each of the villages (nearly a thousand tablets in total), pre-loaded with a custom English-language operating system and SD cards with tracking software on them to record how the tablets were used. Here's how it went down, as related by OLPC founder Nicholas Negroponte at MIT Technology Review's EmTech conference last week:

"We left the boxes in the village. Closed. Taped shut. No instruction, no human being. I thought, the kids will play with the boxes! Within four minutes, one kid not only opened the box, but found the on/off switch. He'd never seen an on/off switch. He powered it up. Within five days, they were using 47 apps per child per day. Within two weeks, they were singing ABC songs [in English] in the village. And within five months, they had hacked Android. Some idiot in our organization or in the Media Lab had disabled the camera! And they figured out it had a camera, and they hacked Android.

So this sounds really inspiring and stuff, even subtracting some obviously sensational stuff (I assume "hacked Android" means "opened up the preferences dialog and flicked a switch"). I've poked around a bit and found similarly fluffy pop-philanthropy articles. Anyone know if there's more reliable information about this out there?

http://www.nsf.gov/news/news_summ.jsp?cntn_id=122523&org=NSF&from=news
From the article:

New research published today in the journal Science suggests it may be possible to use brain technology to learn to play a piano, reduce mental stress or hit a curve ball with little or no conscious effort. It's the kind of thing seen in Hollywood's "Matrix" franchise.

Think of a person watching a computer screen and having his or her brain patterns modified to match those of a high-performing athlete or modified to recuperate from an accident or disease. Though preliminary, researchers say such possibilities may exist in the future.

Experiments conducted at Boston University (BU) and ATR Computational Neuroscience Laboratories in Kyoto, Japan, recently demonstrated that through a person's visual cortex, researchers could use decoded functional magnetic resonance imaging (fMRI) to induce brain activity patterns to match a previously known target state and thereby improve performance on visual tasks.

EDIT: To clarify, this is almost certainly over-hyped. However, it appears to at least be an instance of very interesting biofeedback.

I need help writing a college application essay that will maximize my chances of getting into a school that the world considers prestigious. (17 years old, preparing to enter 12^th grade at a central California high school as of this writing.)

Throughout high school, I resisted being over-scheduled, and basically eschewed all extracurricular activities in favor of having time to think and read. Even when my parents pushed me into things like tennis, dance, or debate clubs (ugh), I was secure in the belief that I could forgo them and rely on my grades and test scores to get me into a college that was good enough to earn a useful engineering degree and find a few interesting friends. (I was right.)

However, my priorities have changed, and I’m starting to really value the extra leverage prestige can bring me. I plan to start a Less Wrong/80,000 Hours club at whatever university I end up attending. I would have access to more intelligent, interested people at Stanford than at, say, UC Irvine. Perhaps more importantly, the club itself would have a better standing in the outside world if it were founded in Stanford. (This in addition to the fact that Stanford already has a world-class Decisions and Ethics Center that may be able to help.)

This is not to say I now regret not being an officer in a dozen useless clubs or participating in endless extracurricular activities. I do, however, regret not doing at least one really impressive, externally-verifiable thing like writing a book. Nothing in my life would make someone say, “Wow, how the hell did she do that?” If admissions officers could scan my brain, they would find a lot that would make them say, “How the hell could she think that?” – but not much of it would be positive.

So my question is, how do I write a personal statement essay, 250-500 words, that will leave an impression in an admissions officer’s mind, without lying or plagiarizing, given that my adolescence was spent thinking and reading, not *doing*? Each university then has 2-4 follow-up prompts (<= 250 words), such as these from Stanford:

1. Stanford students possess intellectual vitality. Reflect on an idea or experience that has been important to your intellectual development.
2. Virtually all of Stanford’s undergraduates live on campus. What would you want your future roommate to know about you? Tell us something about you that will help your roommate—and us—know you better.
3. What matters to you, and why?

The problem with answering these is that all of my *best* answers for these questions (“Newcomblike problems,” “Hey, do you want to join this rationality club I want to start?”, and “optimal philanthropy,” respectively) would take way more than 250 words to explain.

The focus on Stanford, by the way, is because my parents would be extremely unwilling to send me to a university on the East Coast, even if it were really prestigious. But feel free to give me general advice or advice specific to another university. :) If it actually happens, I'll be in a better position to convince them.

May Be Relevant:

I once tutored a girl in Algebra 1 over a period of three months, bringing her grades up from a D to a B. She stopped needing help and I didn’t go looking for another tutee.

I completed NaNoWriMo my freshman year – yeah, it was pretty bad.

I’ve been writing a daily essay on 750 words since December 2010, and have written over 518,000 words in 562 days – writing something 98% of the time, and completing my words 95% of the time. (Although a lot of the missed days were due to glitches in the early website eating my words.)

I entered the Science Fair with a couple friends, hated it because it crushed the spirit of curious inquiry under a predetermined experimental procedure with a predetermined result, and unsurprisingly didn’t win – although we got a certificate from the US Army.

I joined a community service club, hated it because we were just unpaid labor for rich people who didn’t need much help, but stayed anyway because my friends were in it.

General SAT: Reading and Writing scores slightly above the median for most prestigious universities, Math score slightly below. 800's on SAT Math II (Pre-calculus), SAT Biology Molecular, and SAT US History.

5's on AP Calculus AB, AP English Language, and other, less relevant AP's. Five AP classes so far taken, received A's, planning to take 6 more next year.

High probability of a good letter of recommendation from APUSH and Calculus teachers.

Thank you!

Edit: Fixed the hyperlink formatting.

Maybe it would be worth to have a single summary thread for Coursera (and also other source like Udacity etc.) material. At some future point when the courses are on-line and enough people seen them we could work out a "LW curiculum". Here is my subjective list of particularly intersting courses for LW audience:

A Beginner's Guide to Irrational Behavior
Artificial Intelligence Planning
Automata
Basic Behavioral Neurology
Computer Science 101
Clinical Problem Solving
Critical Thinking in Global Challenges
Data Analysis
Fantasy and Science Fiction: The Human Mind, Our Modern World
Game Theory
Human-Computer Interaction
Introduction to Genetics and Evolution
Introduction to Genome Science
Introduction to Mathematical Thinking
Machine Learning
Microeconomics Principles
Model Thinking
Nanotechnology: The Basics
Networked Life
Networks: Friends, Money, and Bytes
Neural Networks for Machine Learning
Neuroethics
Principles of Economics for Scientists
Probabilistic Graphical Models
Quantum Mechanics and Quantum Computation
Rationing and Allocating Scarce Medical Resources
Statistics One
Think Again: How to Reason and Argue

Please note I haven't picked any programming/algorithm courses - there seem to be quite a lot of nice ones. Subscribe here. Plain text list (111 courses):

A Beginner's Guide to Irrational Behavior
A History of the World since 1300
Aboriginal Worldviews and Education
Algorithms, Part I
Algorithms, Part II
Algorithms: Design and Analysis, Part 1
Algorithms: Design and Analysis, Part 2
An Introduction to Interactive Programming in Python
An Introduction to Operations Management
An Introduction to the U.S. Food System: Perspectives from Public Health
Analytic Combinatorics, Part I
Analytic Combinatorics, Part II
Analytical Chemistry
Artificial Intelligence Planning
Astrobiology and the Search for Extraterrestrial Life
Automata
Basic Behavioral Neurology
Bioelectricity: A Quantitative Approach
Calculus: Single Variable
Cardiac Arrest, Hypothermia, and Resuscitation Science
Chemistry: Concept Development and Application
Clinical Problem Solving
Community Change in Public Health
Compilers
Computational Investing, Part I
Computational Photography
Computer Architecture
Computer Science 101
Computer Vision: From 3D Reconstruction to Visual Recognition
Computer Vision: The Fundamentals
Computing for Data Analysis
Contraception: Choices, Culture and Consequences
Control of Mobile Robots
Creative, Serious and Playful Science of Android Apps
Critical Thinking in Global Challenges
Cryptography
Cryptography II
Data Analysis
Design: Creation of Artifacts in Society
Digital Signal Processing
Drugs and the Brain
E-learning and Digital Cultures
Energy 101
Equine Nutrition
Fantasy and Science Fiction: The Human Mind, Our Modern World
Functional Programming Principles in Scala
Fundamentals of Electrical Engineering
Fundamentals of Online Education: Planning and Application
Fundamentals of Pharmacology
Galaxies and Cosmology
Game Theory
Gamification
Greek and Roman Mythology
Grow to Greatness: Smart Growth for Private Businesses, Part I
Health Policy and the Affordable Care Act
Health for All Through Primary Care
Healthcare Innovation and Entrepreneurship
Heterogeneous Parallel Programming
How Things Work 1
Human-Computer Interaction
Information Security and Risk Management in Context
Intermediate Organic Chemistry - Part 1
Intermediate Organic Chemistry - Part 2
Internet History, Technology, and Security
Introduction to Astronomy
Introduction to Finance
Introduction to Genetics and Evolution
Introduction to Genome Science
Introduction to Logic
Introduction to Mathematical Thinking
Introduction to Philosophy
Introduction to Sociology
Introduction to Sustainability
Introduction à la Programmation Objet (in French)
Introductory Human Physiology
Introductory Organic Chemistry - Part 1
Introductory Organic Chemistry - Part 2
Know Thyself
Learn to Program: Crafting Quality Code
Learn to Program: The Fundamentals
Listening to World Music
Machine Learning
Mathematical Biostatistics Bootcamp
Medical Neuroscience
Microeconomics Principles
Model Thinking
Modern & Contemporary American Poetry
Nanotechnology: The Basics
Natural Language Processing
Networked Life
Networks: Friends, Money, and Bytes
Neural Networks for Machine Learning
Neuroethics
Nutrition for Health Promotion and Disease Prevention
Planet Earth
Principles of Economics for Scientists
Principles of Obesity Economics
Probabilistic Graphical Models
Quantum Mechanics and Quantum Computation
Rationing and Allocating Scarce Medical Resources
Scientific Computing
Securing Digital Democracy
Social Network Analysis
Software Engineering for SaaS
Statistics One
The Modern World: Global History since 1760
The Social Context of Mental Health and Illness
Think Again: How to Reason and Argue
VLSI CAD: Logic to Layout
Vaccine Trials: Methods and Best Practices
Vaccines

Post by fellow LW reader Razib Khan, who many here probably know from  the gnxp site or perhaps from his debate with Eliezer.

A few days ago I stumbled upon a really interesting post. And I’m wondering if my readers are at all familiar with the phenomenon outlined here (it was a total surprise to me), The myth of “they weren’t ever taught….”:

With all this I am not saying conditions which are non-hereditary are irrelevant. What I am saying is that we can’t ignore the shape of the pre-existent landscape before we attempt to reshape it to our own image. Excoriating teachers for having pupils who can’t master mid-level secondary school mathematics is in some cases like excoriating someone for the fact that their irrigation canals from the plains into the mountains are failures. You need to level the mountains before your canals can work (or, barring that design and implement a mechanical system which will move water against the grade). Easier said than done. E. O. Wilson said of Communism, “Great Idea, Wrong Species.” The reaction of Communist regimes to this reality was brutal and shocking. Obviously the modern rejection of unpalatable aspects of human nature are not so grotesque. But they have a human toll nonetheless. I’m skeptical that this generation will pass before we have to acknowledge these realities and calibrate our policies accordingly.

Stage One: I will describe this stage for algebra I teachers, but plug in reading, geometry, writing, science, any subject you choose, with the relevant details. This stage begins when teachers realize that easily half the class adds the numerators and denominators when adding fractions, doesn’t see the difference between 3-5 and 5-3, counts on fingers to add 8 and 6, and looks blank when asked what 7 times 3 is.

Ah, they think. The kids weren’t ever taught fractions and basic math facts! What the hell are these other teachers doing, then, taking a salary for showing the kids movies and playing Math Bingo? Insanity on the public penny. But hey, helping these kids, teaching them properly, is the reason they became teachers in the first place. So they push their schedule back, what, two weeks? Three? And go through fraction operations, reciprocals, negative numbers, the meaning of subtraction, a few properties of equality, and just wallow in the glories of basic arithmetic. Some use manipulatives, others use drills and games to increase engagement, but whatever the method, they’re basking in the glow of knowledge that they are Closing the Gap, that their kids are finally getting the attention that privileged suburban students get by virtue of their summer enrichment and more expensive teachers.

At first, it seems to work. The kids beam and say, “You explain it so much better than my last teacher did!” and the quizzes seem to show real progress. Phew! Now it’s possible to get on to teaching algebra, rather than the material the kids just hadn’t been taught.

But then, a few weeks later, the kids go back to ignoring the difference between 3-5 and 5-3. Furthermore, despite hours of explanation and practice, half the class seems to do no better than toss a coin to make the call on positive or negative slopes. Many students who demonstrated mastery of distributing multiplication over addition are now making a complete hash of the process in multi-step equations. And many students are still counting on their fingers.

The author is involved in education personally, so is posting their own reflections as well as what others report to them. In personal correspondence they explain that this phenomenon is common among children of average intelligence. The lowest quartile presumably would never have been able to master many of these rules in the first place. Some of the information resembles the stuff that a friend of mine experienced when he went in to do tutoring for disadvantaged students in Boston when he was getting his doctorate at MIT. At first my friend was totally taken aback at the level of ignorance (e.g., the inability to see the relationship between 1/10 and 10/100). Today he works at a major technology firm as a scientist, but continues to be involved in mentoring “at risk” kids. At some point you have to muddle on. He does his best, and does not indulge in the luxury of shock and disappointment. That helps no one.

This matters because American society is notionally obsessed with education. All this isn’t too clear or important to be frank when you aren’t a parent. It’s somewhat in the realm of the abstract. That changes when you become a parent. Suddenly you become immersed in the data of your local schools, and begin to weight various options to optimize your child’s schooling experience. Of course the real differences in school metrics have not only parental relevance, they matter in terms of national policy and attention. Both the political Left and the Right have their own pet solutions. More money, reform teachers’ unions, charter schools, vouchers, etc.

But the biggest problem at the heart of the matter is the fundamental populist drive to ignore human difference. American schools were designed to produce the citizen, and the citizen has the same rights and responsibilities from individual to individual. In some ways the public school system as it emerged in the 19th century was a project by the Protestant establishment to assimilate white ethnics, in particular Catholics (who of course created their own alternative educational system to maintain cultural separation and distinctiveness). In the 21st century the drive to produce H. Americanus seems quaint, rather, we want to citizens of the world with skills and abilities to navigate an information economy.

What American society on a deep philosophical level, no matter the political outlook, detests acknowledging is that a simple and elegant public policy solution can not abolish human difference. Some children are more athletic than others, and some children are more intelligent than others. Starting among conservatives, but now spreading to some liberals, is a rejection of this premise via blaming teachers. The premise is bewitching because it presents tractable problems with solutions on hand. Here is John B. Watson, the father of behaviorism:

Give me a dozen healthy infants, well-formed, and my own specified world to bring them up in and I’ll guarantee to take any one at random and train him to become any type of specialist I might select – doctor, lawyer, artist, merchant-chief and, yes, even beggar-man and thief, regardless of his talents, penchants, tendencies, abilities, vocations, and race of his ancestors. I am going beyond my facts and I admit it, but so have the advocates of the contrary and they have been doing it for many thousands of years

I think if Watson were alive today he’d have to admit he was wrong. Your ancestors are not destiny, but they are probability. If your father plays in the N.B.A., the probability that you will play in the N.B.A. is not high. But the probability is orders of magnitude higher than if you are a random person off the street.

The Lesswrong community is often a dependable source of recommendations, network help, and advice. When I'm looking for a book or learning material on a topic I'll often try and search here to see what residents have found useful. Similarly, social advice, anecdotes and explanations as seen from the point of view of the community have regularly been insightful or eye-opening. The prototypical examples of such articles are, on top of my head :

http://lesswrong.com/lw/3gu/the_best_textbooks_on_every_subject/

http://lesswrong.com/lw/453/procedural_knowledge_gaps/

the topics of which are neatly listed on

http://lesswrong.com/lw/a08/topics_from_procedural_knowledge_gaps/

And lately

http://lesswrong.com/r/discussion/lw/c6y/why_do_people/

the latter prompted me to write this article. We don't keep track of such resources as far as I know. This probably belongs in the wiki as well.

Other potentially useful resources were:

http://lesswrong.com/lw/12d/recommended_reading_for_new_rationalists/

http://lesswrong.com/lw/2kk/book_recommendations/

http://lesswrong.com/lw/2ua/recommended_reading_for_friendly_ai_research/



math learning

http://lesswrong.com/lw/9qq/what_math_should_i_learn/

http://lesswrong.com/lw/8js/what_mathematics_to_learn/

http://lesswrong.com/lw/a54/seeking_education/


misc learning

http://lesswrong.com/lw/5me/scholarship_how_to_do_it_efficiently/

http://lesswrong.com/lw/4yv/i_want_to_learn_programming/

http://lesswrong.com/lw/3qr/i_want_to_learn_economics/

http://lesswrong.com/lw/3us/i_want_to_learn_about_education/

http://lesswrong.com/lw/8e3/which_fields_of_learning_have_clarified_your/


social

http://lesswrong.com/lw/6ey/learning_how_to_explain_things/

http://lesswrong.com/lw/818/how_to_understand_people_better/

http://lesswrong.com/lw/6tb/developing_empathy/


community

http://lesswrong.com/lw/929/less_wrong_mentoring_network/

http://lesswrong.com/lw/7hi/free_research_help_editing_and_article_downloads/


Employment

http://lesswrong.com/lw/43m/optimal_employment/

http://lesswrong.com/lw/2qp/virtual_employment_open_thread/

http://lesswrong.com/lw/38u/best_career_models_for_doing_research/

http://lesswrong.com/lw/4ad/optimal_employment_open_thread/

http://lesswrong.com/lw/626/job_search_advice/

http://lesswrong.com/lw/8cp/any_thoughts_on_how_to_locate_job_opportunities/

http://lesswrong.com/lw/7yl/more_shameless_ploys_for_job_advice/

http://lesswrong.com/lw/a93/existential_risk_reduction_career_network/

Entertainment

http://lesswrong.com/r/discussion/tag/recommendations/?sort=new

I am considering trying to get a job teaching statistics from a Bayesian perspective at the university or community college level, and I figured I should get some advice, both on whether or not that's a good idea and how to go about it.

Some background on myself: I just got my Masters in computational biology, to go along with a double Bachelors in Computer Science and Cell/Molecular Biology.  I was in a PhD program but between enjoying teaching more than research and grad school making me unhappy, I decided to get the Masters instead.  I've accumulated a bunch of experience as a teaching assistant (about six semesters) and I'm currently working as a Teaching Specialist (which is a fancy title for a full time TA).  I'm now in my fourth semester of TAing biostatistics, which is pretty much just introductory statistics with biology examples.  However, it's taught from a frequentist perspective. 

I like learning, optimizing, teaching, and doing a good job of things I see people doing badly. I also seem to do dramatically better in highly structured environments.  So, I've been thinking about trying to find a lecturer position teaching statistics from a Bayesian perspective.  All of the really smart professors I know personally who have an opinion on the topic are Bayesians, Less Wrong as a community prefers Bayesianism, and I prefer it.  This seems like a good way to get paid to do something I would enjoy and raise the rationality waterline while I'm at it. 

So, the first question is whether this is the most efficient way to get paid to promote rationality.  I did send in an application to the Center for Modern Rationality but I haven't heard back, so I'm guessing that isn't an option.  Teaching Bayesian statistics seems like the second best bet, but there are probably other options I haven't thought of.  I could teach biology or programming classes, but I think those would be less optimal uses of my skills.

Next, is this even a viable option for me, given my qualifications?  I haven't taken any education classes to speak of (the class on how to be a TA might count but it was a joke).  My job searches suggest that community colleges do hire people with Masters to teach, but universities mostly do not.  I don't know what it takes to actually get hired in the current economic climate.

I'm also trying to figure out if this is the best career option given my skillset (or at least estimate the opportunity cost in terms of ease of finding jobs and compensation).  I have a number of other potential options available:  I could try to find a research position in bioinformatics or computational biology, or look for programming positions.  Bioinformatics really makes "analyzing sequence data" and that's something I've barely touched since undergrad; my thesis used existing gene alignments.  I could probably brush up and learn the current tools if I wanted, but I have hardly any experience in that area.  Computational biology might be a better bet, but it's a ridiculously varied field and so far I have not much enjoyed doing research.

I could probably look for programming jobs, but they would mostly not leverage my biology skills; while I am a very good programmer for a biologist, and a very good biologist for a programmer, I'm not amazing at either.  I can actually program: my thesis project involved lots of Ruby scripts to generate and manipulate data prior to statistical analysis, and I've also written things like a flocking implementation and a simple vector graphics drawing program.  Everything I've written has been just enough to do what I needed it to do. I did not teach myself to program in general, but I did teach myself Ruby, if that helps estimate my level of programming talent.  Yudkowsky did just point out that programming potentially pays REALLY well, possibly better than any of my other career options, but that may be limited to very high talent and/or very experienced programmers.

Assuming it is a good idea for me to try to teach statistics, and assuming I have a reasonable shot at finding such a job, is it realistic to try to teach statistics from a Bayesian perspective to undergrads?  Frequentist approaches are still pretty common, so the class would almost certainly have to cover them as well, which means there's a LOT of material to cover. Bayesian methods generally involve some amount of calculus, although I have found an introductory textbook which uses minimal calculus.  That might be a bit much to cram into a single semester, especially depending on the quality of the students (physics majors can probably handle a lot more than community college Communications majors).

Speaking of books, what books would be good to teach from, and what books should I read to have enough background?  I attempted Jaynes' Probability Theory: The Logic of Science but it was a bit too high level for me to fully understand.  I have been working my way through Bolstad's Introduction to Bayesian Statistics which is what I would probably teach the course from.  Are there any topics that Less Wrong thinks would be essential to cover in an introductory Bayesian statistics course?

Thanks in advance for all advice and suggestions!

On blogging websites just as LiveJournal, memes are often in the form of a question or set of questions which a blogger answers in their own blog, then challenges their readers to answer in the readers' blogs (thus spreading).  It doesn't have to be the sort of question to which there is a 'correct' answer.  More usually the meme spreads if the questions are interesting and the answers reveal something about the personality or interests of the blogger.

Here's a recent example:

A ticket for a flight into space (an orbit and a visit to the International Space Station) arrives through the post. Assume that you meet the relevant physical requirements. You can use it, transfer it, or sell it. What do you do and why?

It occurs to me that this sort of thing might be a very accessible way to introduce people to thinking about rationality.   With the example I gave, you get people who would use the ticket because they know that if they didn't they'd always regret not having gone.  However, if the question had been phrased as them receiving money, and a space flight ticket was only one of the things listed as purchasable for that sum, they wouldn't make the same decision.

Suppose LessWrong were to compose a LiveJournal meme.   It would want to be made up of somewhere between 5 and 10 short (paragraph or less) and simple (easy reading comprehension, no obscure terms or prior reading required) questions designed to elicit answers interesting for blog readers to read, intended to introduce bloggers to the concept of a 'cognitive bias' and to thinking about what rationality actually is.

Do you have any suggestions for questions that would work well in such a meme?

TL;DR= There doesn't exist a course/curriculum/general textbook based on Conrad Wolfram's "Computer-Based Math Education" idea. Let's create an open-content one! .... if we can

By computer-based math, I don't mean "math as usual, now taught through a computer!" (a la Khan Academy) I mean "math where we let computers do the calculation drudge-work, while we do the interesting parts."

Or, paraphrasing Conrad Wolfram: "stop teaching kids how to take derivatives; that's what Mathematica^TM is for. Just teach them what a derivative is, so we can move on to more interesting problems. Like, you know, the ones in the real world." (Here's Wolfram's original polemic about the issue.) 

Obviously, this is controversial, and Wolfram spends most of his talk rebutting arguments against it. If, after reading them, you're still not convinced that this is a good idea, then start another thread to discuss it. I don't intend this thread to become a blues-vs-greens battleground. Seriously, just start another thread.

On the other hand, if you are convinced that Wolfram is on to something... 

My problem with this whole venture is that it's too important (IMO) to be left to the Wolframs. 

I mean, come on. Wolfram's basic thesis might be true, but it's no coincidence that this particular truth is being spouted by the brother of the guy who created Mathematica.

And, unfortunately, the Wolframs seem to be the only ones pushing for it. Which means that we won't get any "math, not computation!" courses/textbooks until they can find a taker. 

Now I'm guessing that most LWers would want to reap the benefits of Wolfram's basic idea without having to pay his family a fortune for it, and before however long it takes them to convince an education board about it. (How many "How do I go about learning useful math?" threads have we had so far?) 

So why don't we give the world a leg-up on the path to the widespread mathematical literacy that Wolfram promises? Why don't we put out a computer-based math course for the world?

Obviously, we'd have to use free stuff... Sage instead of Mathematica, for instance. And whatever we put out would have to be free, because... well, if you could write textbooks that people are likely to pay for, you wouldn't need to be part of an LW community venture to do it. 

My major questions, therefore, are:

Are there enough (a) mathematically literate LWers with (b) tons of free time who (c) think computer-based math education is a good cause and (d) are willing to work for free toward a good cause?

I've wondered for some time now what the effects of online education might be on gender and income inequality, specifically as online education interacts with IQ and Conscientiousness (compared with offline education). I ran into a study of a course done online and offline that found correlations with Conscientiousness, which prompted me to start writing out my thoughts: https://plus.google.com/103530621949492999968/posts/aKa3qLatwZ3

The model/argument I give (towards the bottom) is logically trivial, and the basic idea seems pretty intuitive - offline classrooms remove some need for self-discipline/Conscientiousness and performance is more g-loaded - that I'm sure I can't be the first person to think of it.

Does anyone have statistics or citations handy which might help in any essay I write on the topic?

This morning I read an interesting post on the future of education. I thought it would be interesting to have some members of LessWrong discuss it. I know it is idealistic, but some of the points raised were interesting.

Alex Lindsay

The Future of Education

Through an anomaly in the space-time continuum, I fell into the future last week. It was an odd sensation … traveling through time. But at least I made it back. I spent the time I had there at a local school and thought I would share what I saw …

Grades are gone. 

Kids aren't in grade 1 or grade 3 … which was described to me as a "rudimentary" way to "cattle" students. The admins were gentle about it, explaining to me that when school was paper based, there just wasn't the facility to customize the classroom to the student. They explained that even though it was horribly inefficient, they understood why it needed to exist. They did point out that it ran a decade too long, affecting millions … but I changed the subject before it got ugly.

Instead, kids in school have individual achievement levels, which are different for every subject. They have 0-1800 points in each subject. Each student works at their own pace through these milestones and moving forward when they get near perfect scores. A student might have 1500 in one subject and 400 in another. Because everything is online and integrated, there aren't really "grades" like A, B, C, and D … kids are just accumulating points.

When kids max out in a subject, they can spend more time on other subjects; if they are on pace (they are expected to accumulate 100 points a year), they can create independent studies. Many students work very hard to move through the point structure so they can have more free time … which is structured but still up to them. Added resources are applied to students more than 100 points behind their pace. You end up with 20% of the students passing through the system with very little help beyond the structure, 60% getting some help, and 20% getting a much larger amount of attention to move through the system.

Lectures are gone

Lectures the way we know them don't really exist. Most of school is divided into 4 processes: Movies, Games, Projects and Discussion. Movies and Games largely exist on the tablets every student has (these look like iPads but they roll up into a baton-like structure). Projects are done with other students … there are very few opportunities to work on projects alone as it's not seen as an effective character development process in today's job world (where the only people working in a vacuum are doing low-paying work). Discussions are lead by subject experts.

These "Subject Experts" are what used to be called teachers. They are a breed among themselves. They are part brainiac in their field, part Tony Robbins. Their job is to make their subject exciting to learn.

Usually, they begin training for their position very early in life, adding heavy levels of presentation and interpersonal skills to their study load. They work as assistants after reaching 1800 levels in all subjects and focus on a particular subject to master. They train, practice and are allowed to present for basic student events for about decade before they are actually allowed to "solo" an educational subject. It's an incredible amount of work but it also pays well -- salaries for these experts average in what is, in today's money, about $300,000 a year, with the top experts making over a $1M a year. This is largely based on their demand globally. Students are essentially given what is the equivalent to a voucher for discussions and are able to choose their lecturers for each seminar they choose to attend.

There are less of them, as you might guess. Typically about 50,000-100,000 of them at any one time -- much less than the 4M that were working at the peak of the process in the US. While this sounds crazy, we have to remember that most of the objective training is happening interactively within the training tools. There are also over 2,000,000 assistants vying for the Expert positions, providing ongoing support for the students and smaller talks. These assistants are paid, but it's a hard life while they prove themselves.

The discussions are really global events: students attend from all over the world. Some are in theatres together, some are at home, some are in smaller event locations. Students at these events are of all ages. They attend based on their achievement levels, not their age. So you may have 1000 students from 15 countries, aged from 10 to 18. Questions are posted and voted on by the group to percolate to the top and be discussed by the expert (or experts -- there are often people from given industries participating in these events). The events are productions, usually with intense graphics and TV-level production values.

Movies

Top content experts are often the designers and hosts for the online training tools that all students use for their ongoing training. These movies provide core knowledge that is part of the interactive guides that students use to move through their subject matter. These movies are Star Wars-level FX films that explain the subject matter. Some of them are period pieces, some are animated adventures. I'm told as Hollywood stumbled and the education system began to build, many producers moved to this content for survival.

Games

These movies are closely connected to games that the students play -- while they might be something that looks like a geeky version of "Civilizations" or a first person-shooter from the Civil War, or a Physics game that requires students to understand gravity, momentum, etc. The games are not an extra -- they are required and the student scores are connected to their overall achievement scores. These games don't look like the square interactive "Educational" games today. In fact, they have nearly replaced the mindless games of today. I'm told as the government started spending billions on game development, EA and others could A) see that there was money to be made and B) could see there wouldn't be much time to play other games … leading to the new "development gaming" movement.

Projects

Projects are really global affairs. As students reach a required project in a subject (based on their point path), they go online to find others around the world in the same situation. These teams are usually 4-6 people and dig into creating interactive reports that are a mix of video, animation, and text. In addition to deepening subject understanding, the projects are designed to build global relationships and communication. Students are not permitted to do more than 3 projects with the same people in their career. They do rate each other, which builds a bit of a "global team marketplace." Project teams need to have an "Average" score … meaning, high-scoring individuals are encouraged to bring in one or two individuals with lower scores to help them progress. Students are first teamed with "Assistant" mentors, then industry mentors and then industry experts (who are partially in it to recruit students out of school, as their companies pay top dollar for the ability to participate in the "advance" programs, and finding qualified talent has become an incredibly competitive market).

Subjects are slightly adjusted.

Languages (most kids learn English, Chinese, French or Spanish, and an elective language which can things like Japanese, Russian, Arabic or Sign Language). Students are required to be fluent in 4 languages by age 16. Most begin at 4 or 5. A large portion of this training looks like "Rosetta Stone"… then students get into more conversational classes (vocabulary drills are all on the tablets). By age 12, many students are simply taking classes in other languages.

Math - Pretty much the same but with an emphasis on problem solving. There are many fewer equations and more integrated problems. Of course, the students are much more advanced as the more interactive teaching processes have been extremely effective in this area.

Literature - An odd thing to call it given I never saw a book or anything that resembled it. Still, students listen to the classics and discuss the philosophical implications.

Sciences - Kids start in Physics almost at day one. They learn about basic engineering principles in the 300 levels (what could be kind of considered 3rd grade, but it's really what was taught in high school before). 

Global Society (what used to be called Civil Studies and History) - Understanding how cultures around the world evolved to their current state. Understanding one's own state is important, but usually only addressed in the global context.

Personal Development - Now considered one of the most important skills in a highly competitive global jobs market, kids are educated from nearly day one on effective person skills. These skills are not moral or religious, just simply good operating behaviour … and what it takes to be effective.

Creative Arts - From Drawing to Music to Performing Arts … these skills are seen as intrinsic to creating a "Creative" individual that can think their way through the complex issues of the day. In the West, there aren't many "doing-only" jobs that haven't shipped overseas or replaced by technology. As a result, being creative has become much more important.

Physical Arts (what used to be PE) - Ah, the days of Kickball are gone. This is a fairly gruelling daily regime that includes nutrition education and customized exercise processes. Martial Arts, Gymnastics and other dexterity building classes are the norm. Over 25% of the student body globally is a Black Belt. This has more to do with training the mind and self-esteem than person protection.

I asked how this happened in the US … I was told it didn't. In fact, the US was one of the last countries to adopt the still-controversial system. The stakeholders at the time resisted the change and called it too radical to be even tested. The result has been a steep investment to catch up with other countries and much higher unemployment in the US... as many of the information age jobs left the US over the 15 years they resisted the changes.

The revolution actually began in the emerging world, specifically in Africa. New fibre running into East and Southern Africa empowered African nations, with too many kids and not enough teachers, to augment their staff with new videos and interactive learning. The students of these early systems not only learned much faster, but become the most facile at building the content (as they were very familiar with it). I was told as much as 60% of all the content in the global infrastructure is created in Sub-Saharan Africa (Rwanda, Zimbabwe, South Africa and Tanzania).

Anyway, when I arrived back in this time, I wrote this all down as fast as I could to remember it. I hope you find it useful.

Was it really just a dream? I don't know. But if it is was a dream, it was a really good one.

Thoughts? Comments?

tldr; Role-reversal education: by explaining to others, we learn, and improve in confidence.

http://solutionfocusedchange.blogspot.com/2011/12/betacoach-promising-project-to-improve.html

"The project Bètacoach seems to me to be a well thought-out and promising way to give improve math and science education in secondary education. Here is a pointwise explanation of the project:

• What is Bètacoach? In September 2010 a pilot started in which third-graders with low self-confidence in math and science were asked to become coaches of four to five first-graders during math class. Once a week the bètacoaches joined the lesson which was prepared by the teacher, to help their group of first-graders. 
• Role-reversal education: an important principle which is used in the project is that of role-reversal education. Research has shown that by explaning things, people construct knowledge again which helps anchor this knowlegde better and which makes it easier to connect it to information. 
• Choosing bètacoaches: the following steps help to choose the bètacoaches: 1) choose students for whom there is room for improvement with respect to their grades and/or their self-confidence, 2) discuss their suitability for the bètacoach role with your colleagues, 3) aks the students for the role and make clear that the role is an important one, 4) express that you expect that the student will be able to fulfill the role well, 5) be demanding: make it clear that the role requires commitment and effort. 
• Preliminary findings: both the first-graders and the third-graders turn out to be enthusiast about the project. First-graders said the could concentrate better, felt more comfortable to ask questions and understood the material better. The bètacoaches themselves, the third-graders, said they learned from the experience and that they had become more active. "

http://moniquepijls.com/2011/08/16/betacoach-leren-in-een-nieuwe-rol/ (Dutch)

I invite the LW community to help me identify where my thinking about education is wrong here...

While it is necessary to continue to improve our educational systems for youths, we must do a better job of focusing our energies on educating adults, for it is adults who make the decisions about how our society runs, and how we educate our youths. If adults are not sufficiently educated, then youths will not be sufficiently educated.

What kind of educational program do adults require? I advocate a "pull model" of education: adults must be compelled of their own accord to seek knowledge, because learning can at times be quite difficult. Why is it difficult? It challenges our world views, which can be a very scary thing, particularly for people whose world views have remained essentially unchanged for decades.

On the whole, however, the model we currently employ for education is a "push model": we tend to tell people what they must learn in order to be educated. And in the name of efficiency we pre-construct programs of study. But the push model is impractical. It does not account for the range of human experience and desire. To try to force a person to learn probability and statistics, for example, when they have not yet developed a healthy desire to learn it is too stressful and does not allow their minds to be open to reflecting on and integrating the new concepts. The time for a person to learn a given topic is when they begin to ask questions about it.

How can we create an environment that fosters a pull model of education? First, we can offer a series of courses in multiple disciplines that people can sample. This would consist of a broad range of courses in the arts, sciences and humanities. Rather than lasting for several months at a time, they would last for only a few weeks---long enough for the student to get a sufficient introduction to the topic. And they would be fun. For example, people could get a sense of what they can gain from learning probability and statistics by giving them play money at the beginning of a class and having them gamble on some well-constructed games. Then they can be taught some simple, but perhaps non-intuitive mathematical "tricks" that would allow them to be more successful as they play the games.

By sampling a broad range of topics while engaging in discussions about issues facing current society they will both identify things they are talented at and things they would like to see changed in the world. Once these have become clear, with sufficient mentorship the thing they want to focus their studies on will crystallize for them. Once their focus has crystallized, it is time to work with them to build an individualized, interdisciplinary program of study in which everything they study is directly related to what they wish to accomplish, and they understand the connection. Then they will spend some period of time acquiring the skills they need to accomplish their goals.

How should they acquire these skills? By and large we offer only a single model of teaching and learning---lecture, exam, grade (often followed by forgetting; see Father Guido Sarducci's Five Minute University: http://www.youtube.com/watch?v=kO8x8eoU3L4)---although some people are making a lot of money convincing large groups of people that online learning is the way to go, but it appears that few people actually learn well this way. The real point---and people who study education know this, they just don't implement it because the current system is so entrenched---is that people process information in different ways, and we must provide an environment for people to learn in the way they process information.

As people acquire the skills they need, they must put them to use in order to maintain them. This will be done by having them engage in projects that are directly related to what they wish to accomplish. This will also give them practical experience in the world.

What should be the cost of such an education? This depends on the economic situation of the individual. We live in a stressful world. The very people who most need education have very little money to devote to it, even if they desire it. Their education must cost only as much as the individual feels he or she can afford, otherwise they will simply not show up.

Will this create a world in which everyone, or even most people, understand topics like probability and statistics? Maybe, maybe not. But if a large group of people learn even a little more about a wide range of topics, and learn to enjoy and respect learning and knowledge, then we will be far more likely to be living in a more rational world.

Ok, there you go. Have at it.

In school we learn wonderful things like how to find integrals, solve equations, and how to calculate valence electrons of elements based on their atomic numbers. Because, obviously, they will be very important in our futures -- especially if we become artists, musicians, writers, actors, and business people.

We learn so much in school. Yet, when most people look at paintings they don’t truly understand them. When most people listen to music, they don’t really know what they’re hearing. Most people would fail simple music theory tests, even though many have listened to music most days of the week since they were babies!

Similarly, if you have working eyes, you should ask “Why do shadows look like they do? What color is snow, really? Can I predict the colors of different colored materials at different times of the day? If not, why? I have been seeing them for years, haven’t I?”

I think the problem here is that people can’t understand what is really important. Calculus, mechanical physics, chemistry, microiology, etc. are interesting to learn, perhaps. But, they are relatively advanced topics. People don’t use them in daily life unless they are professionals. Why not learn things that we think about every day instead of those that will frankly be useless to most?

Why don’t we learn how to understand our senses?

Learning about sight, sounds, thoughts, etc. should fit in somewhere in the first year of high school. Everyone needs to learn the physics of art and color (e.g. this and this), music theory, rationality, and logic.

For example, why should people start learning (or pretending to learn) philosophy, the art of thinking, in college? Should we be able to make life-changing decisions without even knowing how to spot errors in our thinking?

As a science researcher, I know first hand how hard it is to find a good balance between being well versed in worldly topics and being focused on a field in order to excel in it. But, both of these areas of study should not be called the true basics, in my opinion.

As president of my school's philosophy club, I took a different approach to teaching the basics of philosophy and thinking than traditional classes do. Instead of asking students to discuss the lives and ideas of famous Greek philosophers, I asked them to analyze their own lives and make their own philosophies. As expected, they were terrible at it at first. But, by the end of the year people began to actually think about the world around them.

So, my point is that we should -- in life and in school -- emphasize actual everyday thinking more.

The biggest challenge is that it takes so long!

Purpose of Review

Owen’s recent post brought up the topic of optimizing education. One particular approach, Direct Instruction (Misha’s better explanation), claims to have essentially solved the problem. In particular, Direct Instruction (DI) does allegedly not only work for basic reading skills, but any teaching task. Owen brought up the Michel Thomas language courses as a good application. Language learning is one of my main interests, so I gave the French Foundation course a try.

The main point of the review is to summarize what Michel Thomas actually does, how it differs from other common paradigms and how effective it seems to me.

Summary: Nice for beginners and people with bad learning experiences; limited use afterwards. The audio-only aspect is very convenient. It complements other strategies well and I see it as a good proof-of-concept of DI-like methods for language learning.

Overview

Let’s start with a disclaimer. Michel Thomas (MT) is not officially a DI course and as far as I could google, Thomas propbably wasn’t aware of DI at all. However, according to Solity's The Language Revolution and Owen, the reason MT works so well is that it applies (an approximation of) DI techniques. It is right now the best realistic example beyond the grade school level, so it’ll have to do.

I had some French in high school and thanks to fluency in German and English, I can read some French, but I have no active skill at all, nor have I ever used French in a serious way.

I have now completed the first half of the French Foundation course and skimmed many other courses. You can listen to the first 20 minutes here; they are very representative. Furthermore you can read the booklets to get an idea of the material covered in each course. The whole course is audio-only, consists of 8 CDs (and 2 review CDs) and is intended to be listened to only once.¹ There are several advanced courses which merely cover more grammar points and vocabulary. Structurally, they are all the same.

Method

MT teaches the course to two new students². You’re supposed to take the role of a third student, pausing the recording whenever MT asks a question so that you can say your own answer. One of the two students also answers and you can compare your reply and listen to MT’s advice and error correction. Both students are beginners, so most of your mistakes will be covered that way.

MT introduces one language component at a time and makes you use it in a given sentence. He provides a short explanation first and then lets the students answer a couple of examples by giving them an English sentence and asking them to translate it into French. Each component is thus reinforced through many examples.

MT also tries to combine the translation tasks over time by re-using partial sentences. This way, sentence quickly look complex, but always stay easy. (“What impression do you have of the political and economical situation in France at the present time?” is used about one hour in!)

Vocabulary is only introduced as necessary and relies heavily on cognates. The primary focus is on teaching structure. MT strongly emphasizes not to guess or try to remember anything, but instead to rely on induction (“Do not guess, but think it out!”). This works because the examples are carefully chosen to be as obvious as possible. All translation tasks have only one correct answer. All production is tightly controlled. MT relies on the constant tests to see that the students are successfully keeping up. He is never unsure if some concept has been understood or not.

Complex rules that might thematically belong together (like verb conjugations) are broken apart so that each individual new form or word is learned on its own. Similar rules that might be confused are deliberately spread out.

MT stresses that you aren’t supposed to try to remember anything. If you don’t know something, then he has not succeeded as a teacher yet and he will take care of it, not you. He does this by doing manual spaced repetition, i.e. he repeats previous questions (or similar ones) over time and tests the students constantly. If they have trouble answering, then he quickly goes back to the relevant lesson. This is of course how most language textbooks are supposed to be used, but they rely entirely on the student doing the testing themselves. Instead, MT provides the complete lesson including all necessary repetitions so the student doesn’t have to do anything at all except answer MT’s constant stream of translation tasks. (As a programmer, I’m strongly reminded of loop unrolling.)

What I stood out for me was the reward structure. Students rarely make big mistakes and actual correction is mostly needed for pronunciation issues. The major way students do fail is by simply not remembering something, which MT easily fixes by reminding them again. The students have good confidence in their answers and don’t have to guess. The lessons are fast-paced and consist mostly of tests. MT is constantly positively reinforcing the students, rarely correcting them. The whole lesson looks a lot more like an Anki session than a class room or a traditional textbook.

Comparison to other methods

The course is basically a (minimally edited) live class MT teaches. The result is a very natural pacing. This has the major advantage that it never goes too fast. Most other courses edit out mistakes or necessary repetitions out of fear they might be too boring, but by doing so, no student can actually keep up. This can’t happen with MT’s untrained live students. (Unfortunately, MT’s courses are also unscripted, so he does make a few organizational mistakes and the later courses don’t exactly fit together. Fortunately this is not a big issue due to MT’s large experience.)

A major difference to most other approaches is that MT actively implements what Krashen calls “i+1”, where i is the current level of a learner, meaning that concepts are taught in the order of minimal effort. Each new step contains exactly one new rule. Most language courses group rules according to some underlying pattern, like tenses, and expect you to learn a whole group at once.

MT focuses entirely on production, both by using only translation tasks and by teaching only useful components, i.e. parts of the language you need for a wide variety of contexts. No lesson has only one narrow use. This creates a very active learning experience. I fully agree with this early focus on grammar (but not grammar theory!). Once you’re done with that, you can go more-or-less monolingual and immerse yourself in the target language, relying on spaced repetition software to rapidly build your vocabulary.

Furthermore, MT’s course is very engaging. There is little downtime where you merely listen. It consists almost exclusively of quick tests. Thanks to i+1, you never have to juggle more than one new rule at a time. The subject matter does not get repetitive and MT is a very enthusiastic teacher. This can be a major problem with other language courses.

My main criticism, especially as an autodidact, would be that MT never makes his methods explicit. You entirely rely on him. He may have an awesome lesson plan, but you’re never taught how he arrived at it or how to continue beyond that.³ Hopefully that’s not a general problem with DI. In particular, any language course should teach you how to use spaced repetition. It’s the only sane way to handle vocabulary and prevent unnecessary review sessions.⁴

For contrast, look at the (excellent) Remembering The Kanji, which similarly teaches Japanese characters through decomposition, logical ordering and the use of mnemonics. However, much of the book focuses on teaching the method and the logic behind it, so that you can use it for any amount of characters you want. It is very simple to move beyond the scope of the book. I wish every textbook worked like this.

Outlook

I’m quite impressed by the course design. It’s really effective at building a solid speaking foundation. It won’t get you anywhere near fluency and, being audio-only, totally ignores literacy, but by the end of the course you should have enough skill to actively engage the language.

After finishing MT, you should have a good grasp of the grammar. A good follow-up course might be something like Assimil (video overview), which would take care of literacy and fill in any remaining grammar gaps. After that, the only thing missing is vocabulary and general practice. This is the point where traditional language teaching ends, but graded readers, parallel texts and so on, combined with spaced repetition, solve this problem nicely. Or, you know, start talking, maybe on lang-8.

Personally, I plan to work through the full French, Spanish and Italian courses, and would recommend checking them out. Again, try listening to the preview to see if this approach appeals to you.

Footnotes

A couple of days ago, prompted by several recent posts by Owen_Richardson, I checked out the book "Theory of Instruction" (Engelmann and Carnine, 1982) from my university library and promised to read it this weekend and write a post about Direct Instruction. This is that post.