Support next review date in IR about incremental-reading

[tvhong]

Here are some useful resources related to this:

• SuperMemo wiki youtube: Starting with Incremental Reading by going through the IR Manual
• This add-on's (Incremental Reading) implementation of priority queue: https://github.com/tvhong/incremental-reading/blob/master/ir/schedule.py#L210
• Searching, PDF Reading & Note-Taking in Add Dialog implementation of priority queue: https://github.com/fonol/anki-search-inside-add-card/blob/9688243645c5f3ac7bfe13dd8e76a901e45cd641/src/notes.py#L721
• SuperMemo wiki on Incremental reading for users of spaced repetitions: https://supermemopedia.com/index.php?title=ABC_of_incremental_reading_for_any_user_of_spaced_repetition

[tvhong]

The algorithm for priority queue in SIAC addon (aka Searching, PDF Reading & Note-Taking in Add Dialog) is interesting.

It also supports a single queue for all articles. Simplistically, the algorithm for determine ranking within the queue can be defined as:

# priority between 1-100 with 100 being the highest
# higher rank means higher position in queue 
rank = priority * timeSinceLastReview

This means that between topics with same "timeSinceLastReview" value, ones with higher priority will appear earlier in the queue. And between topics with same priority, ones with longer timeSinceLastReview will appear earlier.

This seems like a fairly good algorithm and clearly has served the author of SIAC very well.

However, in order to correctly compare topics with different priority and timeSinceLastReview, we'll need a bit more calibration. The author of SIAC did this by introducing PRIORITY_SCALE_FACTOR and PRIORITY_MOD configurations. Additionally, they needed special logic for days_delta < 0.5.

# How many times more important is a priority 100 item than a priority 1 item?
PRIORITY_SCALE_FACTOR   : int           = get_config_value_or_default("notes.queue.priorityScaleFactor", 5)

# how should priority be weighted
PRIORITY_MOD            : float         = get_config_value_or_default("notes.queue.priorityMod", 1.0)

def _calc_score(priority: int, days_delta: float) -> float:
    prio_score = 1 + ((priority - 1) / 99) * (PRIORITY_SCALE_FACTOR - 1)
    if days_delta < 0.5:
        return days_delta + prio_score / (PRIORITY_MOD * 10000)
    else:
        return PRIORITY_SCALE_FACTOR * days_delta + PRIORITY_MOD * prio_score

https://github.com/fonol/anki-search-inside-add-card/blob/9688243645c5f3ac7bfe13dd8e76a901e45cd641/src/notes.py#L721-L726

I spent sometime thinking about how we could simplify the algorithm but came up with the exact same 2 configuration values that SIAC introduced. However, from a normal user's point of view, the exact meaning of these 2 configurations are quite mysterious.

At this point, I think it is more difficult to understand the SIAC-like algorithm than the SuperMemo's priority queue algorithm.

[tvhong]

So, here's how I envision how next review date in IR would look like:

• Each IR card has a priority between 1-10 with 10 being the highest (same behavior as before)
• Introduce an "outstanding queue". This queue contains cards to be reviewed each day, sorted by priority.
• Convert the organizer view to be a normal table that can sort by card names or other columns.
• Add a "next review date" column to the organizer view.
• Allow users to edit next review day in the organizer.
• After each review, the user can change the card's priority or just press spacebar to finish the review.
• Un-reviewed cards will be automatically postpone for a future date.
• Cards that are reviewed or postponed have their next review date increased from the last time. How much this increases depends on the card's priority.

Here's a rough outline of the scheduling algorithm:

# priority 1-10 with 10 being highest
newInterval = prevInterval * (1 + 1/priority)

So, for a priority 10 item with prevInterval = 5 days, newInterval = 5 days * (1 + 1 / 10) = 5.5 days.
And for a priority 5 item with prevInterval = 5 days, newInterval = 5 days * (1 + 1 / 5) = 6 days.

And to rank items in the Outstanding Queue, we'll use (priority, lastReviewTime). Items with higher priority (10) will be higher in the queue. If they have the same priority, the one with larger lastReviewTime will be higher. Using lastReviewTime this way, we get the benefits that extracts from the same article will show up in order, and before the main article.

[tvhong]

Describing the above from a user's perspective:

Scenario 1: adding new articles

(This is similar to the current experience)
Import an article and specify the priority of the card.
The card's next review date depends on its priority.

Scenario 2: daily review

Open an IR deck.
Read the first article or extract from the Outstanding Queue.
After finish reading, press "Show Answer".
In the "back" of the card, modify the priority if desired.
Once done, click "Next".
The reviewed card will be scheduled for a future day.
Repeat until all cards in Outstanding Queue is reviewed.
If there are remaining cards by the end of the day, they are auto-postponed to a future day.

Scenario 3: reviewing outstanding queue

Open the outstanding queue via Read > Outstanding.
See a modal with articles and extracts in priority order.
This modal is read-only. To edit, switch to the Organizer view.

Scenario 4: changing next review date or priority

Open Organizer view from Read > Organizer.
Select some cards and click "Change next review date".
Enter new value and save.
If the next review date is today, the card is added to the Outstanding Queue.

Scenario 5: reset interval (not sure if this needs to be an explicit feature yet)

Open Organizer view from Read > Organizer.
Select some cards and click "Reset interval".
This will reset the next review date to 1 and restart the interval calculation

[tvhong]

We can store additional fields into cards from Anki 2.1.55. This might be useful.
ankitects/anki#2039

[tvhong]

I'm implementing this idea in https://github.com/tvhong/incremental-reading/tree/vhong/pq2 . I can get the Outstanding queue and the scheduling working. But cannot show the cards according to the order in the Outstanding queue.

I tried overriding Reviewer._get_next_v1_v2_card and Reviewer._get_next_v3_card in that branch (reference), but that doesn't work because Reviewer keeps track of cards in its internal data structure.

I could probably override getCard method in the v2 scheduler. But the v3 scheduler uses get_queued_cards, which queries data directly from rust backend, is a lot harder to override.

[tvhong]

Checking other addons that change review orders, they only support scheduler v2.

Examples:

• Change Order of Review Cards in Regular Decks - code
• Review in Created Order - code

https://ankiweb.net/shared/info/3731265543

[tvhong]

I'm gonna put this one on hold.

Don't wanna add a new feature that does not work on the latest scheduler.

[tvhong]

Here's a post to ask about how to override review order on Anki's forum: https://forums.ankiweb.net/t/how-to-override-review-order-in-scheduler-v3/23786