[prize] Spaced Repetition literature review

jsalvatier

EDIT: I am canceling this contest because I feel that the structure of the incentives were poorly thought out (see gwern's comments). I will be posting a new better structured contest in the near future (now posted). If you feel this is unfair or otherwise feel slighted, please contact me at my username at gmail.com.

I'm interested in making projects happen on Less Wrong. In order to find out what works and to inspire others to try things too, I'm sponsoring the following small project:

Spaced Repetition is often mentioned on on Less Wrong as a technique for remembering things. I've started using Anki and it certainly seems to be useful. However, I haven't seen a good summary of evidence on Spaced Repetition.

I hereby offer a prize to the first person to submit a good summary of the evidence on Spaced Repetition to the main page. The winner will get the prize, currently: $265 + 40 to charity (see comments)

The summary should address at least the following questions:

What spacing is best?
How much does spaced repetition actually help memory?
Does spaced repetition have hidden benefits or costs?
Does the effectiveness vary across domains? How much?
Is there research on the kinds of questions that work best?
What questions do researchers think are most important?
Is there any interesting ongoing research? If so, what is it on?
What, if any, questions do researchers think it is important to answer? Are there other unanswered questions that would jump out at a smart person?
What does spaced repetition not do that people might expect it to?

The post should summarize the state of current evidence and provide citations to back up the claims in the article.

If you think you would benefit from the result of this project, please add to the prize! You can contribute to the prize on the ChipIn page.

Whether the summary is 'good' will be judged by me. If there is a serious dispute, I'll agree to dispute resolution by any uninvolved party with more than 5k karma.

If you have suggestions, questions or comments, please leave them in the comments.

If you would like to work on this project, please say so in the comments below. Collaboration is encouraged.

This project is tagged with the 'project' tag and listed on the Projects wiki page.

I'm interested in making projects happen on Less Wrong. In order to find out what works and to inspire others to try things too, I'm sponsoring the following small project:

The summary should address at least the following questions:

What spacing is best?
How much does spaced repetition actually help memory?
Does spaced repetition have hidden benefits or costs?
Does the effectiveness vary across domains? How much?
Is there research on the kinds of questions that work best?
What questions do researchers think are most important?
Is there any interesting ongoing research? If so, what is it on?
What, if any, questions do researchers think it is important to answer? Are there other unanswered questions that would jump out at a smart person?
What does spaced repetition not do that people might expect it to?

The post should summarize the state of current evidence and provide citations to back up the claims in the article.

If you think you would benefit from the result of this project, please add to the prize! You can contribute to the prize on the ChipIn page.

Whether the summary is 'good' will be judged by me. If there is a serious dispute, I'll agree to dispute resolution by any uninvolved party with more than 5k karma.

If you have suggestions, questions or comments, please leave them in the comments.

If you would like to work on this project, please say so in the comments below. Collaboration is encouraged.

This project is tagged with the 'project' tag and listed on the Projects wiki page.

From what I remember of the prize literature, prizes work best for well-delineated, objectively judgeable accomplishments. Like putting a rocket into orbit - you can plan out what is necessary, and it's clear when you succeeded. A literature review isn't really either. So I'm not sure a prize in general is succeeded for the task.

My personal problems are that:

I can't bring myself to start a new project like intensively researching spaced repetition to establish exactly how good the evidence is, how effective it is, and what domains it does and does not work in, without having a good idea of what I might receive in turn. Uncertainty is paralyzing for me.
This is especially true because your prize is large - I felt guilty taking a bitcoin from Kiba for Bitcoin is Worse is Better because I knew it was incomplete in several respects, although it's one of my best essays and gave him a lot of traffic; and a bitcoin is only worth $20, so how would I feel delivering something bad for $155? I'd feel really terrible!

The difficult questions are:

How much does spaced repetition actually help memory?

Quantifying the advantage in any more than an ad hoc way like http://www.gwern.net/Mnemosyne#fn3 would be difficult since it requires delving into the specifics of the data, and forgetting curves are non-linear so it's not clear what is even meant by 'how much' - memories per review at 1 year? At 10 years? At 50 years? Integrated over an actuarially projected lifespan?
Does spaced repetition have hidden benefits or costs?

There's probably no direct research on this, so now one has to delve into related research fields and look for very weak evidence or correlations, like maybe high memory correlated with longevity in old people.
Is there any interesting ongoing research? If so, what is it on?

New research tends to be hard to get (= wasted time going around paywalls) and requires following a lot of citations in bibliographies and checking out blogs and emails to really get a sense of where a field is now.

The best way is probably to informally or formally contract with someone like lukeprog (who does this sort of literature review routinely) to conduct the review.

Do you have a good starting point for reading on the prize literature?

2jsalvatier15y

Great comment. Perhaps I should be reading the prize literature! I'll think about how to structure this better. On your personal problems: consider finding someone who has more time and acting as a consultant for them. For example, Duke, who comments elsewhere in this thread. Just to make sure I'm clear, I'm not trying to gain a detailed or very specific understanding of the literature on SR, but to figure out how confident I should be in recommending SR to others and to myself and what I should recommend others and I do and what I should expect it to help and how much. I actually hadn't read your page before, so thanks. I actually think your page is a fairly good start.