LESSWRONG
LW

Comment Permalink

Viliam_Bur12y40

Teaching to the test is bad, but teaching without any feedback is even worse, isn't it?

Sorry if it sounds like a false dilemma; the idea is that we could see any kind of feedback as a "test" and some people will find a way to optimize for this "test". Even if we would define the goal of education as "children are happy and have fun", some teachers would optimize for their students reporting being happy and having fun, even if it is harmful for their education. If the teacher is determined to optimize for what is measured at the cost of the nameless goal, replacing one test with another is just replacing one set of perverse incentives with another set.

I think the long-term solution is inventing more reliable tests. Most likely, that would be a lot of work. For example, if G is fixed, but there is a training that increases the performance in the test, then we could conclude that the test does not measure G well and we need a better test. Making such test is probably horribly expensive; but if IQ is a good predictor of success, it could be money well invested. Perhaps with some smart approach the costs could be reduced. Or maybe one day we will measure the activity of the brain directly, without having to find out the right words to put on the paper.

For education, the fix seems much simpler. If the problem of "teaching to the test" is that students forget their lessons soon after the test, add more tests in longer intervals (Anki-style). If the problem is memorizing without understanding, add more complex questions which are impossible to answer without understanding. If the problem is teachers telling students the specific questions in advance, make a large pool of questions and choose a random subset of them at the last moment. If the problem is teachers helping students during the test, require a presence of an independent person in the room. Etc.

ChristianKl12y00

There are multiple issues with teaching to the test. Take an issue such as writing a political essay.

A student that writes a well argued text that provides other reasons than the ones that are covered in the curiculum will fail. Standardised testing doesn't lend itself for judging critical reflection of material.

If you want to standardised the text you can say, you teach a student knowledge X, Y and Z about the American revolution. Every student that can integrate X,Y and Z get's maximum marks. A student who integrates two passes. A student who only gets ... (read more)

4

Brain Training to maximize returns

11th Oct 2013

1 min read

4

Hi Everyone,

Australia's ABC has recently broadcast a new series called 'Redesign my Brain' with Todd Sampson.

The series seeks to explore how much the brain can be improved in areas like memory and recognition. After just one month of training Todd found himself performing considerably better on tests then he had prior.
He also competed in the World Memorization Championships, and watched a bloke in Germany play 12 games of chess simultaneously without seeing any of the boards.
So other than being a fun show to watch, it got me thinking about the advantages of brain training.
I've had a look at some stuff like dual-n-back, luminosity, and other brain training programs, but I've failed to really explore how much utility such training has.

One of the memory champions was able to remember the order of 25 decks of cards in one hour. But it didn't seem like his ability didn't do much to improve his life beyond providing a fun and enjoyable hobby.

So I'd like to ask:

Which areas of cognitive training do you think would have the best returns in terms of life optimization?

And what do you think would be the best way to go about that training?

Would love to hear some success stories.

4

New Comment

24 comments, sorted by

Click to highlight new comments since: Today at 7:30 AM

Which areas of cognitive training do you think would have the best returns in terms of life optimization?

Exercise. Its effects are well studied, and it has been shown to cause improvements in physical and mental health, cognitive performance, and attractiveness.

[-]Gimpness12y50

I am under the impression that a lot of those Brain Training games you may find won't work as claimed i.e. those (Luminosity etc) that claim working memory improvement --> intelligence improvement/cognitive improvement.

Here is a relevant meta-analysis of such (haven't been able to get an original copy have just read through this article

Tis a shame cause I watched the same program and got excited but decided to investigate before I paid up for it.

http://scholar.google.com/scholar?q=Is%20working%20memory%20training%20effective%3F%20A%20meta-analytic%20review

Melby isn't the only relevant meta-analysis; you can find some more cited in my http://www.gwern.net/DNB%20meta-analysis

Learn math by getting textbooks and doing the exercises.

That seems like more of a screening process than training process; each person has levels of math that they can learn using that method with various levels of effort. Using more effort does not develop any skill other than using effort to study textbooks.

[-]ThrustVectoring12y20

It's worse than that, even. What math textbooks you can pick up and get something out of is determined by your current math background, so it's also a huge element of luck to get books that give you progress instead of discouragement.

I see it as more of a "maybe there is low-hanging fruit and you can teach yourself some math easily, but good luck picking topics you can tackle"

[-]hyporational12y20

Gwern has done some literature review on this subject and pretty much concluded that G isn't trainable.

See also http://www.cambridgebrainsciences.com/

With some training, it was pretty easy for me to double my performance in some of these cognitive tests. I observed no generalized improvement in my cognitive abilities whatsoever.

This reminds me of a common complaint of the US school system, "Teaching to the test."

[-]Viliam_Bur12y40

Teaching to the test is bad, but teaching without any feedback is even worse, isn't it?

Sorry if it sounds like a false dilemma; the idea is that we could see any kind of feedback as a "test" and some people will find a way to optimize for this "test". Even if we would define the goal of education as "children are happy and have fun", some teachers would optimize for their students reporting being happy and having fun, even if it is harmful for their education. If the teacher is determined to optimize for what is measured at the cost of the nameless goal, replacing one test with another is just replacing one set of perverse incentives with another set.

I think the long-term solution is inventing more reliable tests. Most likely, that would be a lot of work. For example, if G is fixed, but there is a training that increases the performance in the test, then we could conclude that the test does not measure G well and we need a better test. Making such test is probably horribly expensive; but if IQ is a good predictor of success, it could be money well invested. Perhaps with some smart approach the costs could be reduced. Or maybe one day we will measure the activity of the brain directly, without having to find out the right words to put on the paper.

For education, the fix seems much simpler. If the problem of "teaching to the test" is that students forget their lessons soon after the test, add more tests in longer intervals (Anki-style). If the problem is memorizing without understanding, add more complex questions which are impossible to answer without understanding. If the problem is teachers telling students the specific questions in advance, make a large pool of questions and choose a random subset of them at the last moment. If the problem is teachers helping students during the test, require a presence of an independent person in the room. Etc.

[-]ChristianKl12y00

There are multiple issues with teaching to the test. Take an issue such as writing a political essay.

A student that writes a well argued text that provides other reasons than the ones that are covered in the curiculum will fail. Standardised testing doesn't lend itself for judging critical reflection of material.

If you want to standardised the text you can say, you teach a student knowledge X, Y and Z about the American revolution. Every student that can integrate X,Y and Z get's maximum marks. A student who integrates two passes. A student who only gets one or zero fails.

If you want to judge a text by a checklist where everyone who has the checklist gets a similar result it's hard to test whether someone can meaningfully think about politics and go beyond just reciting knowledge.

As far as feedback for writing goes I think peer feedback should be encouraged. If a student manages to convince his peers with a political essay the essay is good.

While we are at the topic of peer feedback, the ability to take feedback and revise your text based on the feedback is a very valuable skill that you can't effectively test in standardised testing.

Peer Feedback should also be good for teachers. I would suggest that every teacher spends one hour per week sitting in the classroom of another teacher. At best every week another teacher.

Human's learn best through feedback that doesn't come after a large time gap. I'm not sure that an individual teacher can learn much about improving his own teaching from the standardised test scores.

Instead of teaching to the test, a teacher could try to teach what his students find interesting.

The teacher who teaches to the test doesn't work with the natural curiosity of the children but pushes the children in the direction of the preset goals of a test.

At Sudbury Valley one child might be interested in learning reading at 6 years and another at 10 years. That doesn't mean that one child is smarter than the other. Both can graduate from college. The idea that every child has to learn the same thing at the same age is problematic. It wastes a lot of energy on trying to teach things to children for which they just aren't ready in the moment the curriculum thinks they should learn something.

[-]Viliam_Bur12y00

Instead of teaching to the test, a teacher could try to teach what his students find interesting.

For some kinds of students this would work beautifully, for other kinds of students this would fail horribly.

Also the "students find interesting" part may be impossible to evaluate if students of both kinds are present in the same classroom. Could possibly be solved by teachers announcing lectures on different subjects, and students choosing which lecture they will attend.

The idea that every child has to learn the same thing at the same age is problematic.

I completely agree. It's just easier to organize education this way. Imagine a school with dozen teachers, dozen subjects, a few hundred students, where each student wants to learn different parts of different subjects at different pace.

A part of this mess could be solved by heavy use of computers, but it wouldn't be easy. For example every subject could have defined its "tech tree" (the structure of lecture dependencies), and for each student the computer would remember which lectures they already passed. The teachers would put their schedules in the computer, and each student would see a list of lectures they are ready to attend. But there would be probably a lot of "holes" in the student's schedules, when for all lectures given at this moment the student already attended them, or didn't attend the prerequisities. There could be some activities for these "holes", such as doing a homework, or an unstructured discussion with other students.

(The "tech trees" are necessary, because otherwise there would be a lot of frustration caused by many students choosing a lecture X and then not understanding it because they lack the necessary knowledge Y; especially in the cases where students didn't know that X requires Y, or were overconfident about their knowledge of Y-related things.)

[-]ChristianKl12y00

A part of this mess could be solved by heavy use of computers, but it wouldn't be easy. For example every subject could have defined its "tech tree" (the structure of lecture dependencies), and for each student the computer would remember which lectures they already passed.

SRS can tell you very exactly what a student knows and what he doesn't know. It's much better than looking at which lectures a student attended to know where they are at.

I completely agree. It's just easier to organize education this way. Imagine a school with dozen teachers, dozen subjects, a few hundred students, where each student wants to learn different parts of different subjects at different pace.

Sudbury Valley School is effectively such a place, that's why I used it as example. If you force every school to perform well on yearly standardised tests a school like Sudbury Valley School is practically impossible.

You take a bunch of assumptions about how schools should organise themselves for granted. Standardized testing and curricula force schools to play according to those assumptions. That prevents experimentation with different core assumptions about what school should be about.

Take an issue like emotional management. Even CFAR thinks it's a highly valuable skill to learn. It's not something that a modern standard test where a student tries to score high measures meaningfully.

I could imagine a school where nearly all subjects that get taught are different from the subjects that get taught in our Western curriculum and that still provide kids with meaningful skills for their live.

Reading and basic math could be taught on the side in one our per day in a optimized computer program.

But there would be probably a lot of "holes" in the student's schedules, when for all lectures given at this moment the student already attended them, or didn't attend the prerequisities. There could be some activities for these "holes", such as doing a homework, or an unstructured discussion with other students.

I think we probably want that ever student spends a hour or day on a SRS system anyway. I think looking at Anki 2.X is very interesting. There are so many raw edges and unexplored issues that you could optimize to make it more effective.

We shouldn't live in a world where the mnemosyne database sits around for years without any researcher analysing it and applying some modern math to build a good model around learning.

I really don't understand it. I mean from an academic perspective, the data is there. The mnemosyne guys are open to sharing it. Math talent is there. If you throw good math at learning, it seems like you can get a decent paper out of it. Possibly even a paper that highly cited, when you add some mathematical technique to it that nobody before used on learning.

Calculating SRS repetition dates could be a problem with the same priority as designing computers that play Go or maybe even on the level of protein folding.

The fact that few people care about the problem illustrates well how little interest there is in approaching new ways of learning. I think that lack of diversity between most school is a cause. Getting rid of standarized curricula and standardized testing would open the possibility of schools trying out approaches that differ very much from each other. Different school persuing different concepts then opens up resources to figure out how different ways of learning work.

[-]hyporational12y00

I think lectures could be replaced by videos and interactive material, and teachers could take more the role of an instructor instead of a lecturer. I think it's easier to be a high quality instructor than a high quality lecturer so this kind of centralization of lecturing would be good. Because students would have different strengths and weaknesses, it could be arranged that they also instruct each other.

This is pretty much the premise behind Khan Academy. There's tech trees with motivating level ups, videos, interaction, I think also peer instruction.

[-]Viliam_Bur12y20

I think lectures could be replaced by videos and interactive material, and teachers could take more the role of an instructor instead of a lecturer.

Exactly. Videos are easy to replicate. Humans are interactive. So why not have the best of the both worlds and see the best possible video lecture first, and then discuss it with a teacher?

With the additional advantage that if you miss the school for whatever reason, you could watch the video anyway. Or you could choose to use the same videos for homeschooling.

[-]Brillyant12y10

I did dual n-back for ~60 days. My scores improved gradually to levels that would have seemed rather difficult (beyond what I thought I could do) when I first started. Then I think I got really bored with doing it everyday and my scores leveled off. (It is a really boring exercise...)

Over the course of the 60 days, I developed some techniques that gave me more consistency -- this lead me to believe my brain wasn't getting better generally, as much as I was getting better at dual n-back.

[-]ThrustVectoring12y00

Trying to improve your brain is a bit backwards. You want to acquire and practice the most valuable skill you can. Doing brain training minigames is not the most valuable skill to learn.

It's probably one of the following:

public speaking/small talk (toastmasters)

connecting with your emotions (focusing by eugene gendlin)

a career or trade skill (trucking school for short turnaround, apprenticeship program in the trades)

typing (if you are below 50 wpm and write significant amouts, it's hours saved per day. If you write ludicrous amounts, then stenographic typing can help make the process flow better as it keeps up with your thoughts.)

communication (particularly nonviolent communication, there are youtube videos)

comfort zone expansion

I'm probably missing a lot of stuff that I've internalized to the point to where it's hard to imagine not having the skill. Even bringing up the last sentence reminded me of the skill of speaking standard or technical english.

[-]buybuydandavis12y00

There was a google tech talk a while back which I've managed to track down: http://www.youtube.com/watch?v=UyPrL0cmJRs

http://www.positscience.com/why-brainhq/brain-training-your-way

His basic theme was that intelligence was made up of the combination of very tiny functional skills that could be trained and improved separately. Anyway, he claims to have demonstrated improvement. Maybe. The method appeals to me and makes some sense. Say that improving your ability to distinguish phonemes would reduce cognitive load for other tasks farther up the conceptual processing pipeline. Should help, just a matter of how much.

[-]ThrustVectoring12y10

There are surprising amounts of cognitive load in visual processing, too. Getting your vision done and organizing your living and working spaces can help a surprising amount.

[-][anonymous]12y-20

If all my sensory information and thoughts were demonstrably as accessible and accurate as the best of my sensations and thoughts, I could be more successful at all I do (and hire myself out as an expert witness).

Pending that, amping up gut feelings / body language reading to effective mind reading. Or mind control of others.

[-]BaconServ12y-30

If this were a trivial task, LessWrong wouldn't exist.

What if it were a problem with a retrospectively trivial solution?

[-]BaconServ12y-10

The problem appears to be underspecified: Trivial by post-G-increase standards or trivial by current standards?

If it's by current standards, then we should invest in figuring out the pattern of trivial details that go disproportionately unnoticed. I'm not sure how we can use information before we've obtained it, unless I'm missing something obvious about causality. The logic of not being able to act on that realization before I've had it seems air-tight to me, unless it's a tool you can use without realizing it, in which case I'm not sure why I haven't used it. It might be possible to use it without being aware you have, but that doesn't solve the problem of my not consciously knowing it before I've realized it.

If it's not by current standards, then it would depend on what "trivial" means in terms of a being significantly more generally intelligent than myself. If "trivial" in that sense has any relation to being ably to easily explain a concept, then such an intelligence should be able to communicate the idea to us, even though it may be non-trivial by our current understanding.

Replaying memories of conversations with agents of lesser immediate intelligence (Read: Knowledge? Understanding?) than my then-self, the only trivial explanation I've been able to determine is that there is no shortcut to G-increase. Once you stop looking for secret backdoors to decoding the mechanics of reality, you're liable to suffer less distractions in doing so. In unsure if this qualifies as obvious or trivial or what the total cognitive relatedness of the two concepts is.

If there is an easy button, it is trivial by our standards to recognize it retrospectively: "This is the button such that people who press it get smarter."

I thought you were saying that there is no easy button, not that any easy button is difficult to recognize.

All that said, I think one of the major optimizations that many people could do is to perform VOI calculations before looking for an easy way; the return-on-work for looking for easier ways of doing something is often very low (in cases where lots of people have already looked), or very high (in cases where few people have already looked), but it seems that the actual distribution is the reverse.

[-]BaconServ12y20

Ah, that helps me understand what went wrong in the communication of my sentiment. Thank you for clearing up that inferential silence.

More from Goobahman

Curated and popular this week