Improve learning algorithm for empty pool.

Coordinator
Mar 19, 2012 at 2:01 PM

This thread is to continue the discussion from HERE.

Mar 23, 2012 at 8:23 PM

To recap the discussion:

The concept of "spaced repetition" is that each time the learner demonstrates knowledge of a certain piece of information, this piece of information is repeated less often. In other words, well-remembered information is reviewed less often than information that is not well-known to the learner.

https://www.memorylifter.com/about/scientific-approach.html

MemoryLifter does this beautifully until the pool is empty.  At that point, the whole concept is abandoned in favor of sequentially asking the cards based on the timestamps.  (As NYCGuy said so succinctly, the programmer made the unwarranted assumption that the pool will never be empty.)  Christoph explained that the LearnLift team was concerned that cards in box 10 would never be asked, and their contents forgotten.  While that may be true, cards that are not known at all (in box 1) or barely known (in box 2 or other low boxes) are asked only once through an entire cycle of cards, at the same rate as those in high boxes.

Of course, it was suggested that I tell the program to learn a specific box, or change box sizes to force the program to work.  That is unacceptable.  When I'm making 1000+ card modules for my homeschooled children to use to learn their Latin and Spanish vocabulary, they can't be relied on to notice when the pool is empty and to change the box sizes, then change them back when they add another chapter to learn.  So I came up with some ideas:

  • As long as the pool contains cards, do what is currently done.
  • If both the pool and box 10 are empty, read from the lowest box.  It is only the cards in box 10 that can never be promoted.  All others need more practice.
  • Once the pool is empty and box 10 contains at least one card, several possible solutions are available:
  1. Choose a number.  Maybe you could even allow the user to change it if he doesn't like your default.  Ask that many cards from the lowest box, then one card from box 10 for review.  Or, if you prefer, only count correct answers from the lowest box before reviewing a card from box 10.
  2. Choose a number.  If the user can change it, "zero" should be an acceptable choice.  Review that many cards from box 10, then one card from the lowest box.  Or only count correct answers in the review.  [Actually, this fails "spaced repetition," as more cards would be asked from box 10 than from lower boxes.]
  3. Alternate between one card in box 10 and one card in the lowest box.
  4. Don't worry about it, because eventually all the cards will be in box 10 (barring typos and brain-cramps), and that's all that will be available to practice.
  5. Here's a new one:  if a card is promoted to box 10, review the first one in box 10.  That might be harder to code, to make sure the program differentiates between a card already in box 10 and a card promoted, but would provide for review whenever a card floats up.

I'm sure other ideas can be found.  Whatever is implemented needs to follow the concept of "spaced repetition."  Just because someone has practiced everything at least once, doesn't mean he knows everything equally well.  And while review is a good idea, we really, really need to practice the cards we don't know well (or at all) much more than we need to review.

Coordinator
Mar 24, 2012 at 11:32 AM
MrsPan wrote:

Of course, it was suggested that I tell the program to learn a specific box, or change box sizes to force the program to work.  That is unacceptable.  When I'm making 1000+ card modules for my homeschooled children to use to learn their Latin and Spanish vocabulary, they can't be relied on to notice when the pool is empty and to change the box sizes, then change them back when they add another chapter to learn.

Thank you very much for the great input and summary of your ideas (which I think most of them would be a good solution!), but I think the quoted way "would" be the best way, but of course you are right and you can't change that manually every time and that is the reason we already tried a "auto box size algorithm" once, but it didn't make it into a final version. I need to check if I find the algorithm and the results (and why we kicked it out again)...

So to everyone who reads this, please give us your oppinion which way you think is best, as MemoryLifter is a community driven project now, "we" want to do what the community wants and not what we think is best anymore.

Jan 15, 2014 at 1:23 PM
I was just checking in to see if this "Empty Pool" problem has been solved, or not. After a few years of not using MemoryLifter, I am now looking at Flashcard programs to brush up on studying some foreign languages. MemoryLifter has been my first choice in the past, but I too ran into issues with this "Empty Pool" issue. So, in response to your request for our opinions, here is mine.

The last time I used MemoryLifter heavily was when I was in Nursing School. I had thousands of terms to memorize. When I think of what I needed then, is that MemoryLifter work just as described as if we were using a manual box of cards, promoting cards from the pool to box 1, then box 1 to box 2, box 2 to box 3, etc... In reality, I was learning, and adding, so many new medical words/terms per week, that I never got any cards promoted to box 10. When I had half an hour to sit down and study with MemoryLifter, I wanted the program to start with the Pool, then go on to Box 1, Box 2, etc.... These were always the newest cards and/or the cards that had been demoted. My time was limited, and those lowest cards were that ones that needed studying for the next test.

Unfortunately, like the original poster noted, when my Pool was empty, I discovered that MemoryLifter started asking cards all over the place. This was a problem for me because I really did not need to study cards in Box 5, for example, that may have been on a test 3 weeks ago. I really needed to concentrate and spend my limited study time on the lowest boxes with the newest terms that were coming up for the test this week. So I would manually go into MemoryLifter and tell it I wanted to study Box 1. I also manually changed the size of the boxes in an attempt to force MemoryLifter to work the way I needed it to work, asking the cards in the lowest boxes first. Yes, I could force the program to help me study which cards I really wanted to study, but it was always a bit frustrating that such a great program seemed to have a logic problem.

In summary, when I was in Nursing School and I had limited time to study, I would rather MemoryLifter continue to ask the cards in the lowest boxes, promoting/demoting the cards as required. I did not care about the "spaced repetition" concept because those older cards where not going to be on my next test.

Having said that, now that I am studying some foreign language vocabulary just for my own benefit, and I am not worried about the "next test," so the "spaced repetition" concept makes more sense to me for life long learning.

It would be great if we could chose which learning option we wanted MemoryLifter to use if the pool was empty - lowest boxes first, or oldest cards first. If that is not possible, my vote would be to have the program work its way from the lowest box up.
Jan 20, 2014 at 4:15 AM
Edited Jan 20, 2014 at 4:15 AM
cfi wrote:
MrsPan wrote: Of course, it was suggested that I tell the program to learn a specific box, or change box sizes to force the program to work.  That is unacceptable.  When I'm making 1000+ card modules for my homeschooled children to use to learn their Latin and Spanish vocabulary, they can't be relied on to notice when the pool is empty and to change the box sizes, then change them back when they add another chapter to learn. Thank you very much for the great input and summary of your ideas (which I think most of them would be a good solution!), but I think the quoted way "would" be the best way, but of course you are right and you can't change that manually every time and that is the reason we already tried a "auto box size algorithm" once, but it didn't make it into a final version. I need to check if I find the algorithm and the results (and why we kicked it out again)... So to everyone who reads this, please give us your oppinion which way you think is best, as MemoryLifter is a community driven project now, "we" want to do what the community wants and not what we think is best anymore.
First, the best solution to the empty pool "problem" is simply to avoid the empty pool. People learning languages should have no problem at all finding new material, even in flashcard format, for studying. One can always try to "stay ahead" of his/her needs by having available study material before it is actually needed. For example, the home schooling parent should develop the learning material before the children need it.

That being said, the best approach to the empty pool would be to proceed as follows:
Once the pool is empty, treat the second box -- the one where newly promoted cards normally go -- AS IF it was the very first one, and stop using the first one.
If necessary, then treat the 3rd box, as if it was the first one, and stop using the second (and first) box...etc.

To give more detail, imagine a 4-box system, with box sizes 20, 50, 70 and 100. You have gone through you last batch of cards, and both the first box and the pool are empty (nothing to learn). Now you pretend that your system only has 3 boxes (of sizes 50, 70 and 100). So, you treat the 50-sized box (your initial box 2) as you normally did the 20-sized box (which now you don't use). You study the cards in the 50-sized box, cycling through them and promoting those you remember to the 70-sized box, (former box 3 now playing the role of a box 2). As you review the upper boxes you demote cards to the 70-sized box (remember, your former 20-sized box-1 no longer counts).

Of course, if you keep this up, eventually your current box 1 (former box 2) may also go empty. At this point you ignore box 2, and pretend your system consists of only two boxes of sizes 70 and 100 (these were originally your boxes 3 and 4). You now treat your 70-sized box (initially your box 3) as you normally treat a box 1. Etc., etc.

Eventually you would be down to a one-box system of capacity 100.

Notice that in all cases, cards promoted from the last box should NOT be demoted to the current (or former) box 1. The reason is that you still want to focus on the cards you know least, and those promoted from the final box are precisely the ones you know best (that is why they made it all the way to the last box and you still remember them!). A better way to handle cards promoted from the last box, is simply to add yet another (larger) box to your system (say sized-250 in this example), where you can put such cards. This way, you keep these cards mostly out of the way, but eventually you may review them if/when your new final box is full (at which point you could add yet another, even larger, final box, and so on).


Virgil
P.S. Too bad Leitner's book has never been translated from German to English (a translation to Spanish is out of print).