Hi everyone :). I like Anki, find it quite useful and use it daily. There is one thing that constantly annoyed me about it, though - the state of shared decks and of infrastructure around them.

There is a lot of topics that are of common interest for a large number of people, and there is usually some shared decks available for these topics. The problem with them is that as they are usually decks created by individuals for their own purposes and uploaded to ankiweb. So they are often incomplete/of mediocre quality/etc and they are rarely supported or updated.

And there is no way to collaborate on the creation or improvement of such decks, as there is no infrastructure for it and the format of the decks won't allow you to use common collaboration infrastructure (e.g. Github). So I've been recently working on a plugin for Anki that will allow you to make a full-feature Import/Export to/from JSON. What I mean by full-feature is that it exports not just cards converted to JSON, but Notes, Decks, Models, Media etc. So you can do export, modify result, or merge changes from someone else and on Import, those changes would be reflected on your existing cards/decks and no information/metadata/etc would be lost.

The point is to provide a format that will enable collaboration using mentioned common collaboration infrastructure. So using it you can easily work with multiple people to create a deck, collaborating for example, via Github, and then deck could be updated and improved by contributions from other people.

I'm looking for early adopters and for feedback :).

The ankiweb page for plugin (that's where you can get the plugin): https://ankiweb.net/shared/info/1788670778

Github: https://github.com/Stvad/CrowdAnki

Some of my decks, on a Github (btw by using plugin, you can get decks directly from Github):

Git deck: https://github.com/Stvad/Software_Engineering__git

Regular expressions deck: https://github.com/Stvad/Software_Engineering__Regular_Expressions

Deck based on article Twenty rules of formulating knowledge by Piotr Wozniak:

https://github.com/Stvad/Learning__How-to-Formulate-Knowledge

You're welcome to use this decks and contribute back the improvements.

This is a follow-up to last year's report. Here, I will talk about my successes and failures using Spaced Repetition Software (SRS) in the classroom for a second year. The year's not over yet, but I have reasons for reporting early that should become clear in a subsequent post. A third post will then follow, and together these will constitute a small sequence exploring classroom SRS and the adjacent ideas that bubble up when I think deeply about teaching.

Summary

I experienced net negative progress this year in my efforts to improve classroom instruction via spaced repetition software. While this is mostly attributable to shifts in my personal priorities, I have also identified a number of additional failure modes for classroom SRS, as well as additional shortcomings of Anki for this use case. My experiences also showcase some fundamental challenges to teaching-in-general that SRS depressingly spotlights without being any less susceptible to. Regardless, I am more bullish than ever about the potential for classroom SRS, and will lay out a detailed vision for what it can be in the next post.

This is a detailed description of my reading and learning workflow. You may find ideas to adopt, or maybe you can tell me what I could be doing differently!

Overview

I've been using Spaced Repetition on and Off for the past few years, and have built a solid Anki habit this last three months, to the point where now I wonder how I could read books without entering the important points into Anki.

I recommend getting a habit of using Spaced Repetition, it's a small habit that doesn't require too much willpower (it can feel like a game, if done right!), and is useful in the long term.

Daily routine: transit

I have a dozen or so Anki decks. Some I consider “valuable” (Algorithms, Driving Code, Git commands), some less so (Paris Metro, Hiragana and Katakana, Vim commands, …). I also carry around a book, notebook and four-color pen.

On any downtime (waiting for transit, waiting in line in a store, standing in crowded transit…), I’ll review my decks, starting with those with the most due cards.

On some days I may not finish all the decks, but that’s no big deal; with an hour and a half of transit per day, I’ll get to them eventually.

If I can sit for a bit of time, and don’t have too many outstanding cards, I’ll usually read a book (or work on stuff in my notebook if I have some stuff that needs brainstorming).

Reading books

If I’m reading fiction, I’m relaxing, I don’t need to try to remember anything :)

If I’m reading non-fiction, I’ll usually have an index card as a bookmark and place to take notes - things to look up, summaries and rephrasings, diagrams, page numbers of parts to come back to, and of course things to enter in Anki (though I’ll sometimes just directly enter them in my phone).

I’ll reread my notes when I finished the book or a big chapter, or when I come back to the book after a long time, and eventually enter them in Anki (usually with Anki's web interface, which is quicker than typing on a phone).

Reading online material

I have a bunch of Google Docs where I take notes on various topics (why Google Docs? I can search them, share them if needed, work with them from various places). If I’m reading something I want to remember, I’ll usually have a corresponding google doc open in another window (so I can see both at the same time - hunting through tabs breaks the flow). My notes will be a mix of

• URLs marked as “to read” or “read” (with maybe a summary of what it’s about)
• Verbatim quotes
• Rephrasings, insights, questions, brainstoriming
• “anki format” cards (pairs of question, then answer), for example, from my Haskell deck:

How do I declare that Integer is of class Eq, using IntegerEq?
instance Eq Integer where
  x == y                =  x `integerEq` y

(note that in this case it's three lines, when entering into Anki I'll have to put the first line as question and the two other ones as answer)

Building the anki cards in Google docs makes it easier to make related cards by copying and pasting the same question and changing little bits ("Question: ???, B and C", "Question: A, ??? and C", "Question: A, B and ???")

In the evening, when I don’t have the energy for something more difficult, I’ll occasionally copy batches of stuff from Google Docs into Anki. To do that first I copy everything into a plain text file (to strip all formatting, otherwise things look weird in Anki and it’s distracting), and then cut-paste the cards into Anki by alt-tabbing between the text file and the Anki web interface (this sounds cumbersome but can be done fairly quickly using pretty much only the keyboard).

What if I get behind?

No big deal, I’ll review the “important” decks first, and then eventually catch up on the rest (Some people recommend using one big deck for everything; I prefer having several small decks because it makes it easier to catch up with what matters if I “fall of the bandwagon”).

What I learned

• Make Stupid and easy cards; I aim for having answers that are a single word
• I delete or suspend cards that I suspect are a waste of time (because I don’t care about learning that; because it’s too difficult; because I suspect it’s wrong).
• Double-sided cards are useful for learning languages (I used to make both directions independently)
• If you're learning a foreign language with a weird alphabet, it's worth the extra effort of finding an imput system on your phone (or computer) that handles that alphabet.

What I’d like to improve

Batch-entering data is a bit complicated, I wish I could just select a bunch of text in google docs and say "just put all this in Anki". However, as a low-energy habit batch-copying stuff feels a bit like a game so I don't mind that much.

• I wish I could put some decks at “low throttle” and some at “high throttle” (say, I want to learn 20 driving code cards a day, but only 3 vim cards). Anki has a setting that says how many new cards you get, but it's global; so either I change that setting all the time (which can be done fairly quickly), or control the influx by leaving stuff in Google Docs.
• I wish I could control randomization: just select a bunch of cards and say "randomize these". There's some cards I want to see in a random order, and some where I'd rather see them in the original order.
• Anki is bad at handling synchronization, if I used Anki on my phone and want to use the web interface, I need to synchronize first, which takes a few minutes and may fail; otherwise there will be a conflict and I will have to pick which of the two datasets I keep. This is another reason why I prefer to use Google Docs for staging: waiting for synchronization breaks my flow.
• How do people use evernote or supermemo?

More resources on Spaced Repetition

The article on the Wiki points to a few discussions here of Spaced Repetition (which are worth reading if you want to see how other people use it), including Gwern's excellent article.

How about you? Do you use Spaced Repetition? Have you tried, but give up? Do you have a workflow with some bits that differ from mine? Do you have any tips of things I could do better?

Added February 2014: As I no longer have time to keep both posts updated, all further updates will be made to the version of this post hosted on my personal blog.

In a recent post, Qiaochu Yuan noted that "various mnemonic techniques like memory palaces, along with spaced repetition, seem to more or less solve the problem of memorization." The list below is an attempt to compile all existing Anki decks created by Less Wrong users, in the hope that they will be of help to others in memorizing the corresponding material. (Anki is arguably the most popular spaced repetition software.)  If you know of a deck not included here, please mention it in the comments section and I'll add it to the list. Thanks!

This post is about two Anki plugins I just wrote. I've been using them for a few months as monkey patches, but I thought it might help people here (or at least the 20% that are awesome enough to use SRSs) to have them as plugins. They're ugly and you may have to fiddle for a while to get them to work.

1. Music-Fiddler

To use this, play music while doing Anki revs. (I also recommend that you try playing music only while doing Anki, as a way of making Anki more pleasant.) While you're reviewing a card, the music volume will gradually decrease. As soon as you pass or fail the card, the volume will go back up, then start gradually decreasing again. So whenever you stop paying attention and instead start thinking about all the awesome things you could do if only you were able to sit down and work, the program punishes you by stopping the music. And whenever you concentrate fully on your work and so go through cards quickly, you have a personal soundtrack!

To use this plugin:

- If you do not have Linux, you'll need to modify the code somehow.

- Ensure that the "amixer" command works on your computer. If it doesn't, you're going to need to modify the code somehow.

- Make sure you have the new Anki 2.0.

- Download the plugin.

- Change all lines (in the plugin source) marked with "CHANGEME" according to your preferences.

- You might want to disable convenient ways of increasing the volume, like keyboard shortcuts.

This plugin provides psychological reinforcement, but is not proper intermittent reinforcement, because it is predictable and regular instead of intermittent. I'm not sure whether this should be fixed; I haven't yet gotten around to trying it with only intermittent volume increases.

2. Picture-Flasher

After answering a card, this plugin selects, with some probability, a random image from a folder and flashes it onto your screen briefly. This gives intermittent reinforcement.

To use this plugin:

- I haven't tested it on non-Linux operating systems, but I can't see any obvious places it'll fail.

- Make sure you have the new Anki 2.0.

- Get pictures from someplace; see below.

- Download the plugin.

- Change all lines (in the plugin source) marked with "CHANGEME" according to your preferences. Be sure especially to put in your picture directory and the number of pictures you have.

To get pictures, I downloaded high-scoring pictures off of reddit. This script can do that automatically. You can use pictures of cute animals, funny captioned pictures of cats, or more questionable things.

The plugin could be made a lot more awesome by having it automatically pull pictures from the internet so you're not reusing them. I'm not planning on doing this anytime soon (because I have no internet on my main computer for productivity reasons), but if somebody else does that and posts it, they are awesome and they should feel awesome.

Update 4 Dec: Emanuel Rylke has created a patch for this plugin which removes the requirement to rename the pictures. It also moves the configuration options to the top of the plugin, making them easier to find. The new version is at the same download link

Update 16 June 2015: The plugins were deleted from the official list where they previously were, apparently because my AnkiWeb account was deleted due to disuse. So I've uploaded the two plugins on GitHub here: https://github.com/StephenBarnes/AnkiPlugins. I also re-uploaded the plugins to the official list. Links on this post have been updated.

Spaced repetition systems use math to determine the optimal way to study things. This post is about an idea I've been trying for a few months for improving SRS for some subjects.

SRSs usually use a pretty rigid system of asking questions and demanding answers. I think that for many subjects it's not very important to know specific answers, either because such answers can be looked up easily or because the gist of a subject is more important. So here's an idea: add an ebook to a spaced repetition system and read/skim each chapter or page when it's due for review. This can be used for ebooks, physical books, or articles from the internet or elsewhere.

For books or ebooks, there are two ways to do this: either add each page as an individual card (with an image of the page right on the card) or create a card for each section or chapter. The latter technique can be used for non-electronic books. If each page is its own card, you can review things more quickly because you don't have to open an ebook or book each time you review, but you'll need to convert the ebook to images first. You can also add annotations, either by editing page images, typing notes onto pages' cards, or adding annotations with your ebook-reading software.

One way to convert ebooks to images is to use imagemagick. On Linux,

convert -density 180x180 BOOK.pdf folder/imgname.png

Change the density if images are too small or too large. You'll have to convert ebooks to pdf format first. This command creates all the pages as imgname-1.png, imgname-2.png, etc. Move the images into a .media folder where your other anki decks are. Use a script to make a card for each page. For example, using python:

f = open("bookcards", "w")

for i in range(0,NUMPAGES): f.write(str(i+1) + ";<img src=\"imgname-" + str(i) + ".png\" />\n")

You probably want to review cards in the order they were created (so that you'll review due cards by page number). This option doesn't exist in anki, so you'll need to make each book a separate deck and use the patch command to apply this diff to /usr/share/anki/anki/deck.py, or wherever that file is on your computer:

64a65
> REV_CARDS_CREATED_FIRST = 4
389c390,391
<                 "priority desc, factId, ordinal")[self.revCardOrder]
---
>                 "priority desc, factId, ordinal",
>               "created asc")[self.revCardOrder]
3557a3560,3561
>           'createdDesc':
>           '(created desc)',
3566a3571,3572
>       if self.revCardOrder == REV_CARDS_CREATED_FIRST:
>           required.append("createdDesc")
4507a4514
>       4: _("Review in ORDER CREATED"),

Also, for each deck, go Settings->Advanced->Initial button intervals and set them so there's no randomness.

Pros of this technique:

• You can add lots of content quickly.
• You can use an SRS to learn things that you wouldn't be able to otherwise. How would you add Godel, Escher, Bach to an SRS in question-answer format?
• Reading books without memorizing them is silly unless your aims in reading do not require long-term retention of the books' contents.
• You can keep the context of facts, and you automatically preserve the original phrasing.

Cons of this technique

• The spacing is probably not optimal.
• Review is passive, not active. Active recall has been shown to improve memory. This technique trades away memory-detail for time (and other things).
• You may make irrelevant associations between things just because they're next to each other in the book.
• Reading through things takes long. If you skim through your due cards, then you might miss things. You'll also have to skim elementary explanations again (perhaps that's a good thing) or suspend them.
• It takes time to convert books to images, or to open a book or ebook each time you need to review it.

Thoughts?

As people have likely forgotten, jsalvatier had generously set up a contest to look through the literature on the spacing effect. Duke submitted his entry a while ago, and I turned mine in on the 1 August deadline. It's part of my overall essay on spaced repetition: gwern.net/Spaced repetition#literature-review

There's a lot of research, and I only cover what I estimate to be 0-5% of the extant literature, so if you're interested only in the punchline: gwern.net/Spaced repetition#review-summary

(This is only a pointer to the review because the formatting would drive me nuts on LW. I'm not sure the edit box even supports footnotes at all; nor do I intend to find out.)

Update: the prize is now finished!

The previous contest was poorly formatted for eliciting the most useful reviews of the spaced repetition literature so I've created a new slightly different contest. 

I'm interested in making projects happen on Less Wrong. In order to find out what works and to inspire others to try things too, I'm sponsoring the following small project:

Spaced Repetition is often mentioned on Less Wrong as a technique for adding facts to memory. I've started using Anki and it certainly seems to be useful. However, I haven't seen a good summary of evidence on Spaced Repetition and I would like to change that.

I hereby offer a prize, currently $385, to the best literature review submitted by August 1st. 'Best' will be judged by voting with discussion beforehand by the Seattle LW meetup group. People are not allowed to vote for their own submissions.

The summary should address questions such as:

• What spacing is best?
• How much does spaced repetition actually help memory?
• Does spaced repetition have hidden benefits or costs?
• Does the effectiveness vary across domains? How much? 
• Is there research on the kinds of questions that work best? Especially for avoiding 'guessing the password' and memorizing the card per se rather than the material.
• What questions do researchers think are most important?
• Is there any interesting ongoing research? If so, what is it on?
• What, if any, questions do researchers think it is important to answer? Are there other unanswered questions that would jump out at a smart person?
• What does spaced repetition not do that people might expect it to?

The post should summarize the state of current evidence and provide citations to back up the claims in the article. Referencing both academic and non-academic research is encouraged. Lukeprog's The Science of Winning At Life sequence contains several examples of good literature review posts.

If you think you would benefit from the result of this project, please add to the prize! You can contribute to the prize on the ChipIn page.

If you have suggestions, questions or comments, please leave them in the comments. Prizes demotivating? Due date too soon/far? Specification too vague? Judgement procedure not credible enough?

This project is tagged with the 'project' tag and listed on the Projects wiki page.

I'd like to figure out how to create habits more effectively and systematically, especially mental habits. 

EDIT: I am canceling this contest because I feel that the structure of the incentives were poorly thought out (see gwern's comments). I will be posting a new better structured contest in the near future (now posted). If you feel this is unfair or otherwise feel slighted, please contact me at my username at gmail.com.

I'm interested in making projects happen on Less Wrong. In order to find out what works and to inspire others to try things too, I'm sponsoring the following small project:

I hereby offer a prize to the first person to submit a good summary of the evidence on Spaced Repetition to the main page. The winner will get the prize, currently: $265 + 40 to charity (see comments)

The summary should address at least the following questions:

I have made some significant progress in organizing myself with org-mode (basically a really well thought out emacs outliner) - consider this a plug :). 

Now I think I am ready to bite the bullet and automate another part of my mental apparatus, memorization. I'd like to hear other people's experiences with SRS - spaced repetition - (negative, too), what software they use, what do they use it for, how much time they spend. I expect these to vary, so stating your reasons is worth an extra upvote (and thanks ahead)

ETA: When do you decide something is worth memorizing vs. putting it into a searchable database?

I suspect that forgetfulness is the single largest hindrance to me improving my rationality. This isn't something I've seen others report on LessWrong, so I'm suspicious that I'm in some kind of self-serving spiral, or that I'm doing something obvious wrong. So, I'm seeking feedback on (a) whether the above statement is true -- whether forgetfulness is likely to really be a dominant hindrance; (b) what I can do about it; and (c) why others haven't reported this.

Ways that I suspect forgetfulness harms me:

• I forget past experiences that would otherwise allow me to observe correlations and extrapolate time-series style. Did my mood improve last time I called this friend? Was I more productive during that period where I got less sleep?
• I fail to recall evidence that would be salient when evaluating probabilities. This includes evidence relevant not only to coffee-time discussions on abstract topics, but also questions that occur to me about akrasia and the like.
• I can't remember names of authors and papers that could be relevant to ideas that arise during my research. Searching online for papers is fine for major literature reviews, but enormously expensive to evaluate hour-by-hour conjectures that occur to me.
• When I encounter the same problem multiple times, I forget how I solved the problem last time.
• When a new productivity/social/rationality strategy *does* work, I forget to keep using it. This isn't only about laziness: sometimes I actually wonder explicitly whether strategy X worked, and I can't remember.
• I forget names, faces, places, facts and figures. But I understand this one is quite common.
• I forget all the ways that forgetfulness frustrates me.

Steps I've taken:

• I keep an elaborate diary with appointments like "bring USB drive home from work" and "purchase bread en route to Sam's house".
• I keep an elaborate e-notebook where I try to record my "brain state" so that I can more quickly pick up where I left off with my work.
• Every time I solve a technical problem, I write it down.
• I use a memory app called mnemosyne to memorize foreign language words, names of jitsu techniques, etc.

Spaced repetition - like the 'Anki' program does - is one of the most efficient ways to learn new things. (For research citations, see 'Study methods', here.)

I previously explained how to get up and running with Anki on an Android phone. Here's the guide for using Anki on a Mac:

1. Go here and click 'Download Latest Release' (or here if you're on PowerPC).
2. Open the downloaded .dmg file, drag the Anki icon into your Applications folder in the dock, and then run the Anki application. (Click 'Open' when warned that the application was downloaded.)
3. Click 'Download' and search the list of decks for 'Less Wrong'. Download the 'Less Wrong Sequences' deck.
4. Set your review options as desired, then click 'Review.'

Spaced repetition is one of the most efficient ways to learn new things. (For research citations, see 'Study methods', here.)

The best way to practice spaced repetition is to install Anki to your phone, since you have your phone with you all day long.

I have an Android phone, so here's my 60-second guide to getting started with Anki on Android:

1. On your Android phone, open 'Market.'
2. Search for 'Anki'.
3. Install the 'AnkiDroid Flashcards' app.
4. In your app drawer, run 'AnkiDroid'.
5. It will prompt that you don't have any decks downloaded. Tap 'Download deck' and choose 'Shared decks.'
6. It will take a while to bring up the list of decks available online. Search for 'Less Wrong' and you'll see the deck called 'Less Wrong Sequences.' Download it.
7. Go back to the AnkiDroid main screen, choose 'Load other deck.' Choose 'Less Wrong Sequences.'
8. Set your options for 'New cards per day', 'session limit (minutes)', and 'session limit (questions)', then tap 'Start Reviewing.'