A couple of days ago, prompted by several recent posts by Owen_Richardson, I checked out the book "Theory of Instruction" (Engelmann and Carnine, 1982) from my university library and promised to read it this weekend and write a post about Direct Instruction. This is that post.
Learning through examples
Direct Instruction is based on a theory of learning that assumes the learner capable of extracting a concept inductively through examples of that concept. I may not know what a blegg is, but after you show me several examples of bleggs and rubes, I will be able to figure it out. The principle of DI is to use the same basic procedure of giving examples to teach every concept imaginable. Naturally, in some cases, the process might be sped up by giving an explanation first; furthermore, there are some things in every subject you just have to memorize, and DI doesn't magically change that. However, it is assumed that the examples are where the real learning occurs.
The meat of the theory is using experimental data and cognitive science to establish rules for how examples ought to be given. Here are a few of the more basic ones:
- It is impossible to demonstrate a concept using positive examples alone. Here I am reminded of the 2-4-6 game, in which subjects fail to test triplets that disconfirm their hypothesis. A teacher has control over the examples presented, so it is important to disconfirm the hypotheses that the learners (consciously or unconsciously) generate.
- To successfully teach a quality, it is important that all positive examples only share that one quality. Imagine that you are being taught what a blegg is by a sequence of examples that include blue eggs and red cubes. By the end, you will not be certain whether the defining feature of a blegg is that it's blue, or that it's an egg, or both at once, or if the critical factor is the vanadium ore content of an object.
- The way the example is presented is also a quality that must be controlled in this fashion. This is because inductive learning is not entirely a deliberate process on the part of the learner. For instance, if positive and negative examples alternate, the learner may extract the rule that "every other object is a blegg". There are multiple ways this can become a real problem: I've encountered calculus students who were confused by a problem that asked them to integrate with respect to a variable called "t", rather than "x".
- The examples must be followed by tests, which fall in the range of given examples but are not identical. This is the way to diagnose the learning process, and is the reason that you get ideas such as "DI is about asking the students 10 questions a minute." This is not a defining feature of DI, but you can see now that it can easily happen when the concept being taught is a simple one.
I don't mean to imply that DI is restricted to dealing with yes-or-no identification questions. The examples and concepts can get more complicated, and there is a classification of concepts as comparative, multi-dimensional, joining, etc. This determines how the examples should be presented, but I won't get into the classification here. In practice, a lot of concepts are taught through several sequences of examples. For instance, teaching integration by substitution might first involve a simple sequence of examples about identifying when the method is appropriate, then a sequence about choosing the correct substitution, before actually teaching students to solve an integration problem using the method.
Faultless communication
"Faultless communication" isn't a misnomer exactly, but I think it lends itself to some easy misconceptions. The basic idea is that a sequence of examples is a faultless communication when there is only one possible rule describing all the examples; there is then the often-repeated statement that if a faultless communication fails, the problem is with the learner, not with the method.
When the book gets into details, however, the actual theory is much less dismissive. In fact, it is emphasized that in general, when a method fails, there's something wrong with the method. A well-designed sequence of examples is not (usually) a faultless communication. Rather, it is a sequence of examples calibrated in such a way that, if the learner arrives at an incorrect rule, the test examples will identify the incorrect rule, which can then be traced back to an ambiguity in the examples given. Alternatively, it can make it clear that the learner lacks sufficient background to identify the correct rule.
The actual issue that the concept of faultless communication is meant to address is the following. When you don't have a clear way to diagnose failure while teaching a concept, it leads to blind experimentation: you ask "Did everyone understand that?" and, upon a negative answer, say "Okay, let me try explaining it in some different way..." You might never stumble upon the reason that you are misunderstood, except by chance.
My own thoughts
A disclaimer: I have very little experience with teaching in general, and this is my first encounter with a complete theory of teaching. Parts of Direct Instruction feel overly restrictive to me; it seems that it doesn't have much of a place for things like lecturing, for instance. Then again, a theory must be somewhat restrictive to be effective; unless the intuitive way I would teach something is already magically the optimal way, the theory is no good unless it prevents me from doing something I would otherwise do.
An interesting aspect of Direct Instruction that I don't think has been pointed out yet (well, the book, written in 1982, might not be a likely place to find such a thought): this method of teaching seems ideally suited for teaching an Artificial Intelligence. Part of the gimmick of Direct Instruction is that it tries, as much as possible, not to make assumptions about what sort of things will be obvious to the learner. Granted, a lot of the internal structure still relies on experimental data gathered from human learners, but if we're creating an AI, it's a lot easier to program in a set of fundamental responses describing the way it should learn inductively, than to program in the concept of "red" or "faster than" by hand.
I still have the book and plan to hold on to it for a week or so; if there are any questions about what Direct Instruction is or is not, ask them in the comments and I will do my best to figure out what the theory says one way or the other.
Most excellent Gwern!
I have a proposition!
So, I've begun writing a new post, “A dry presentation of some empirical evidence on DI's effectiveness”. (An attempt to replace that intended function of my original post with as high-quality a replacement as Misha's post was for the intended function of the 'theory sketch' section.)
KPier very kindly offered to help me with editing, so I sent her the first seven-ish paragraphs I had written. She found one change to recommend, somewhat ambivalent herself over which way was best. I wasn't sure either, and found myself wondering what I'd decide in the end.
Then I started wondering about what differences in responses there might be between a post where she made all the final decisions, and a post where where I did.
And then I thought... double-blind experiment! (Woot woot, raise the empirical roof. :P)
Here's my idea:
I finish writing the post, get the 'her final' and 'my final' versions, and then make a post linking to both versions and explaining the experiment.
I'll just label them version A and version B (flip a coin to avoid any weird bias I may have on As and Bs, not that I'd anticipate much), and ask the reader to follow one or the other (by flipping a coin to avoid any weird bias they may have; Mostly just to make sure the sample sizes for each version are equalized.)
Then people record their impression and give me their feedback (without directly quoting the text), and I have to try and discriminate which readers got which.
Does that sound like a neat idea? If it works well, it seems like it might even end up being worth creating an automated system for setting up and running such experiments (without all the coin flipping and link following), for people to use with appropriate posts.
It's been done before, but not often, so I infer it doesn't work well. Possibly this is just due to clumsy implementation.