Spaced repetition (SRS) for learning

Question

A client has asked me to add a simple spaced repeition algorithm (SRS) for an onlinebased learning site. But before throwing my self into it, I'd like to discuss it with the community.

Basically the site asks the user a bunch of questions (by automatically selecting say 10 out of 100 total questions from a database), and the user gives either a correct or incorrect answer. The users result are then stored in a database, for instance:

userid  questionid  correctlyanswered  dateanswered
1       123         0 (no)             2010-01-01 10:00
1       124         1 (yes)            2010-01-01 11:00
1       125         1 (yes)            2010-01-01 12:00    

Now, to maximize a users ability to learn all answers, I should be able to apply an SRS algorithm so that a user, next time he takes the quiz, receives questions incorrectly answered more often; than questions answered correctly. Also, questions that are previously answered incorrectly, but recently often answered correctly should occur less often.

Have anyone implemented something like this before? Any tips or suggestions?

Theese are the best links I've found:

• http://en.wikipedia.org/wiki/Spaced_repetition
• http://www.mnemosyne-proj.org/principles.php
• http://www.supermemo.com/english/ol/sm2.htm

Answer 1

What you want to do is to have a number X_i for all questions i. You can normalize these numbers (make their sum 1) and pick one at random with the corresponding probability.

If N is the number of different questions and M is the number of times each question has been answered in average, then you could find X in M*N time like this:

• Create the array X[N] set to 0.
• Run through the data, and every time you see question i answered wrong, increase N[i] by f(t) where t is the answering time and f is an increasing function.

Because f is increasing, a question answered wrong a long time ago has less impact than one answered wrong yesterday. You can experiment with different f to get a nice behaviour.

The smarter way A faster way is not to generate X[] every time you choose questions, but save it in a database table. You won't be able to apply f with this solution. Instead just add 1 every time the question is answered wrongly, and then run through the table regularly - say every midnight - and multiply all X[i] by a constant - say 0.9.

Update: Actually you should base your data on corrects, not wrongs. Otherwise, questions not answered neither true nor false for a long time, will have a smaller chance of getting chosen. It should be opposite.

Answer 2

Anki is an open source program implementing spaced repetition. Being open source, you can browse the source for libanki, a spaced repetition library for Anki. As of Januray 2013, Anki version 2 sources can be browsed here.

The sources are in Python, the executable pseudo code language. Reading the source to understand the algorithm may be feasible. The data model is defined using sqlalechmey, the Python SQL toolkit and Object Relational Mapper that gives application developers the full power and flexibility of SQL.

Answer 3

Here is a spaced repetition algorithm that is well documented and easy to understand.

Features

• Introduces sub-decks for efficiently learning large decks (Super useful!)
• Intuitive variable names and algorithm parameters. Fully open-source with human-readable examples.
• Easily configurable parameters to accommodate for different users' memorization abilities.
• Computationally cheap to compute next card. No need to run a computation on every card in the deck.

https://github.com/Jakobovski/SaneMemo.

Disclaimer: I am the author of SaneMemo.

import random
import datetime

# The number of times needed for the user to get the card correct(EASY) consecutively before removing the card from
# the current sub_deck.
CONSECUTIVE_CORRECT_TO_REMOVE_FROM_SUBDECK_WHEN_KNOWN = 2
CONSECUTIVE_CORRECT_TO_REMOVE_FROM_SUBDECK_WHEN_WILL_FORGET = 3

# The number of cards in the sub-deck
SUBDECK_SIZE = 15
REMINDER_RATE = 1.6

class Deck(object):

    def __init__(self):
        self.cards = []

        # Used to make sure we don't display the same card twice
        self.last_card = None

    def add_card(self, card):
        self.cards.append(card)

    def get_next_card(self):
        self.cards = sorted(self.cards)  # Sorted by next_practice_time
        sub_deck = self.cards[0:min(SUBDECK_SIZE, len(self.cards))]
        card = random.choice(sub_deck)

        # In case card == last card lets select again. We don't want to show the same card two times in a row.
        while card == self.last_card:
            card = random.choice(sub_deck)

        self.last_card = card
        return card


class Card(object):

    def __init__(self, front, back):
        self.front = front
        self.back = back

        self.next_practice_time = datetime.utc.now()
        self.consecutive_correct_answer = 0
        self.last_time_easy = datetime.utc.now()

    def update(self, performance_str):
        """ Updates the card after the user has seen it and answered how difficult it was. The user can provide one of
        three options: [I_KNOW, KNOW_BUT_WILL_FORGET, DONT_KNOW].
        """

        if performance_str == "KNOW_IT":
            self.consecutive_correct_answer += 1

            if self.consecutive_correct_answer >= CONSECUTIVE_CORRECT_TO_REMOVE_FROM_SUBDECK_WHEN_KNOWN:
                days_since_last_easy = (datetime.utc.now() - self.last_time_easy).days
                days_to_next_review = (days_since_last_easy + 2) * REMINDER_RATE
                self.next_practice_time = datetime.utc.now() + datetime.time(days=days_to_next_review)
                self.last_time_easy = datetime.utc.now()
            else:
                self.next_practice_time = datetime.utc.now()

        elif performance_str == "KNOW_BUT_WILL_FORGET":
            self.consecutive_correct_answer += 1

            if self.consecutive_correct_answer >= CONSECUTIVE_CORRECT_TO_REMOVE_FROM_SUBDECK_WHEN_WILL_FORGET:
                self.next_practice_time = datetime.utc.now() + datetime.time(days=1)
            else:
                self.next_practice_time = datetime.utc.now()

        elif performance_str == "DONT_KNOW":
            self.consecutive_correct_answer = 0
            self.next_practice_time = datetime.utc.now()

    def __cmp__(self, other):
        """Comparator or sorting cards by next_practice_time"""
        if hasattr(other, 'next_practice_time'):
            return self.number.__cmp__(other.next_practice_time)