Comments (31)
That's not obvious. Maybe you should add text explaining that, and an example? You can't expect most people to know that.
from fsrs4anki.
I will do it tomorrow.
from fsrs4anki.
@Expertium, do you have the code to draw the workload graph?
from fsrs4anki.
I do, but beware that for large decks and large values of "Days to simulate" it will take a lot of time to generate the graph, even if we limit the range to 0.7-0.99 instead of 0.5-0.99.
Plotting a graph from Sherlock's data.zip
EDIT: the upper limit for the y axis was hard-coded, I made it adaptable. I also changed the lower limit of the x axis from 0.5 to 0.7 and added text to make it more clear what the horizontal dashed lines mean.
EDIT 2: I want to make the colors adaptable too, for example, the green area should cover everything from min. workload to 2x min. workload. But that's going to be a bit complicated, give me some time.
from fsrs4anki.
Plotting a graph from Sherlock's data.zip
Here's the code. However, with the way I defined the green area, it may be possible for the output of "Compute optimal retention" to fall within the yellow area.
EDIT: there are still some edge cases. Give me some more time.
EDIT 2: ok, I covered all edge cases I could think of, though the code is a bit complicated as a result. I've also deifned the green area differently to decrease the chances that the optimal value will fall into the yellow area.
from fsrs4anki.
For example, if 90% retention is only 5% more work than 75% retention, but you get questions wrong 2/5ths as often, then that is much better.
By the way, "Compute optimal retention" doesn't minimize the workload, rather, it maximizes the sum of all probabilities of recall of all cards. In simple terms, it maximizes knowledge acquisition within given time constraints.
from fsrs4anki.
For example, if 90% retention is only 5% more work than 75% retention, but you get questions wrong 2/5ths as often, then that is much better.
By the way, "Compute optimal retention" doesn't minimize the workload, rather, it maximizes the sum of all probabilities of recall of all cards. In simple terms, it maximizes knowledge acquisition within given time constraints.
ah thank you, I took that understanding from what someone said on reddit. That sounds more sensible. I've realised the values I'm getting from compute optimal retention were because I had a large deck size (explaining the difference between what I'm managing to do, and what the algorithm spit out). I was entering a deck size figure by what I'd expect my deck size to be by the time of exams, as it can't account for a varying deck size. Maybe I'm more interested in having the compute optimal retention feature gain like a linear interpolation feature where there is the option to linearly interpolate between current deck size, end deck size using the number of days of study.
My retention for an hours worth of Anki per day in 180 days rapidly declines between 1000 deck size and 2000 deck size from around 90% retention to more the higher 70s. (but I think that might be due to in part not accounting for the cards that have already matured etc)
from fsrs4anki.
Added in the optimizer:
from fsrs4anki.
Thanks!
My two arithmetic decks (where I fail a card if I didn't answer it quick enough) got graphs where 98-99% is best, so the whole graph is red.
Compared to the values stored in Anki, Anki's Evaluate shows that optimizer's RMSE is usually a little higher, and Log Loss a little lower. (corrected: usually, not always)
from fsrs4anki.
Can you show that graph here?
from fsrs4anki.
imp 0 veryimp mathbas 12%.apkg
[3.1008, 3.1008, 8.3177, 25.1118, 5.3022, 1.7371, 1.1355, 0.0697, 1.6848, 0.1068, 1.1423, 1.5057, 0.1995, 0.4903, 0.5435, 0.0127, 3.9198]
Loss before training: 0.2054
Loss after training: 0.2014
imp post math-bas 14%.apkg
[1.1157, 2.5695, 8.9803, 20.0419, 4.9463, 0.7911, 0.7905, 0.0424, 1.6001, 0.1579, 1.0157, 2.0435, 0.1945, 0.3917, 1.3053, 0.1832, 3.161]
from fsrs4anki.
Huh. I guess the default values of "Days to simulate" and other stuff just aren't very good for you.
@L-M-Sherlock what are the defaults? 365 days, 10 000 cards, 30 minutes of study per day? Maybe you should add an option to configure all of that, just like in "Compute optimal retention" in Anki itself
from fsrs4anki.
@L-M-Sherlock what are the defaults? 365 days, 10 000 cards, 30 minutes of study per day?
So it still depends on the choice of time of study?
Past year:
imp 0 veryimp mathbas 12%.apkg:
Days studied: 77% (281 of 364)
Total: 2508 reviews
Average for days studied: 8.9 reviews/day
If you studied every day: 6.9 reviews/day
Total: 2 hours
Average for days studied: 0.5 minutes/day
If you studied every day: 0.4 minutes/day
Average answer time: 3.2s (18.74 cards/minute)
imp post math-bas 14%.apkg:
Days studied: 65% (237 of 364)
Total: 1099 reviews
Average for days studied: 4.6 reviews/day
If you studied every day: 3.0 reviews/day
Total: 1 hours
Average for days studied: 0.3 minutes/day
If you studied every day: 0.2 minutes/day
Average answer time: 4.03s (14.88 cards/minute)
from fsrs4anki.
Huh. I guess the default values of "Days to simulate" and other stuff just aren't very good for you. @L-M-Sherlock what are the defaults? 365 days, 10 000 cards, 30 minutes of study per day? Maybe you should add an option to configure all of that, just like in "Compute optimal retention" in Anki itself
Would it be possible to set the default number of cards, to the number of cards the user currently has in their deck? I imagine that might be a more sensible default than 10k.
from fsrs4anki.
Maybe you should add an option to configure all of that, just like in "Compute optimal retention" in Anki itself
Actually, the current implementation is configurable: https://github.com/open-spaced-repetition/fsrs-optimizer/blob/bddc4fe67184fdcfaf79284b0e5430a11fcb3c31/src/fsrs_optimizer/fsrs_optimizer.py#L1220-L1227
from fsrs4anki.
But I don't see those options in fsrs4anki_optimizer.ipynb.
from fsrs4anki.
You can pass the arguments into the function in fsrs4anki_optimizer.ipynb.
from fsrs4anki.
So this graph is much better than trying different values to get individual numbers and then thinking about them, but it suffers from all the other problems of Evaluate.
I opened the stats for "imp 0 veryimp mathbas 12%.apkg"', "deck:current introduced:100", and it shows that there have been 3 reviews per day (2 with -rated:1), which is about 8.7 seconds.
There are 569 new cards.
figs = optimizer.find_optimal_retention(deck_size=569, learn_span=365, max_cost_perday=9)
figs = optimizer.find_optimal_retention(deck_size=569, learn_span=365, max_cost_perday=6)
Just over 140 cards memorized in the year.
I introduce only 1 card per day, so it should be 569 days. And I will probably have backlogs again, so I try changing 6s to 5s.
figs = optimizer.find_optimal_retention(deck_size=569, learn_span=569, max_cost_perday=5)
But apparently 6s was already not enough, so I can learn a new card less often than every other day if I am to fit in 5s?
It seems that the Average answer time is 3.2-3.5s, not 2.92 that I used for some reason.
from fsrs4anki.
And I will probably have backlogs again, so I try changing 6s to 5s.
max_cost_perday
is in minutes, and it's not per card, it's per all cards of that day.
from fsrs4anki.
1800 seconds = 30 minutes, 1800 minutes won't fit in a day.
from fsrs4anki.
Ok, my bad. I looked at the code again, and it appears to be in seconds. But I'm still pretty sure that it's per all cards, not just one card.
This would've been much easier if only it was properly documented in the .ipynb itself.
from fsrs4anki.
I was talking about the time per review that I multiplied by the number of reviews (since Anki rounds per-day stats down to 0).
from fsrs4anki.
Ah, so you're plotting a graph for a "What if I only learn 1 card per day?" scenario? I don't think this feature is intended to be used that way, but ok.
from fsrs4anki.
Ah, so you're plotting a graph for a "What if I only learn 1 card per day?" scenario? I don't think this feature is intended to be used that way, but ok.
Yes, I introduce 1 new card per day in that deck. It's not 1 review per day.
As I understand it,
- Optimizer limits overall repetitions. It introduces as many new cards as it can fit in max_cost_perday. It doesn't support new_cards_limits, but it can't review overtime.
- Simulator limits overall repetitions + obeys new_cards_limits. It can't review overtime.
- Anki can limit overall repetitions and obey new_cards_limits. You can review overtime. But the only built-in way to get a dynamic new card limit like with Optimizer or Cardistry is to limit reviews and review overtime by choice.
from fsrs4anki.
- Optimizer limits overall repetitions. It introduces as many new cards as it can fit in max_cost_perday. It doesn't support new_cards_limits, but it can't review overtime.
- Simulator limits overall repetitions + obeys new_cards_limits. It can't review overtime.
I don't think simulator obeys new card's limits, actually. But I'm not sure.
from fsrs4anki.
Where? I don't see it in https://github.com/open-spaced-repetition/fsrs-optimizer/blob/bddc4fe67184fdcfaf79284b0e5430a11fcb3c31/src/fsrs_optimizer/fsrs_optimizer.py or in https://github.com/open-spaced-repetition/fsrs-optimizer/blob/bddc4fe67184fdcfaf79284b0e5430a11fcb3c31/src/fsrs_optimizer/fsrs_simulator.py.
from fsrs4anki.
I'm looking at the simulator too
from fsrs4anki.
https://github.com/open-spaced-repetition/fsrs4anki/blob/main/fsrs4anki_simulator.ipynb
from fsrs4anki.
Ah, that one. Yeah, I was looking at the one that is used for calculating optimal retention
from fsrs4anki.
Simulator needs a switch to disable Anki simulation.
Optimizer needs to support a new card limit.
Still not sure what to do about the existing review cards, since they are causing the most workload in the decks where I am not learning new cards.
For the above deck of 569 cards, with 1 new card/day in 600 days and pretty much no review limits (100 reps, 60 seconds), I got
DR, time, remembered cards, time per remembered card (rem. = remembered cards÷569):
remembered cards is int(card["retrievability"].sum())
, and then the rounded value is used to calculate time per remembered card.
0.01 46.5 201 0.23 (anki: 569 180.7 560 0.32)
0.02 46.5 202 0.23 (rem. 0.355)
0.05 47.2 203 0.23 (rem. 0.357)
0.08 48.2 206 0.23 (rem. 0.362)
0.09 51.1 211 0.24 (rem. 0.371)
0.10 53.0 203 0.26 (rem. 0.357, anki: 569 181.4 559 0.32)
0.20 90.6 269 0.34 (rem. 0.473)
0.30 129.4 334 0.39 (rem. 0.587)
0.40 140.6 411 0.34 (anki: 569 180.3 559 0.32)
0.50 134.7 456 0.30 (anki: 569 178.8 560 0.32)
0.60 122.5 481 0.25 (anki: 569 176.6 560 0.32)
0.70 119.9 511 0.23, 2 leeches
0.75 113.6 525 0.216 (rem. 0.923)
0.83 115.2 535 0.225
0.84 112.5 536 0.201
0.84 113.9 535 0.213
0.84 115.6 537 0.215
0.85 114.0 539 0.212 (rem. 0.947)
0.85 114.6 539 0.2127 (no internal rounding)
0.85 114.7 539 0.213
0.85 115.7 540 0.214 (no internal rounding)
0.85 115.2 539 0.214
0.85 116.7 536 0.218 — with the FSRS values I had before today
0.86 115.5 539 0.214
0.86 115.6 539 0.214
0.86 117.4 540 0.216, 1 leech
0.87 116.9 541 0.216
0.87 118.2 540 0.219
0.87 119.2 540 0.221, 1 leech
0.88 119.2 542 0.22
0.89 118.1 543 0.218
0.92 127.6 552 0.231 (rem. 0.970)
0.99 306.1 566 0.54 (rem. 0.995)
1.00 604.4 24 25.18 (unlike the other lines, only 25 cards were "learned" at all)
114÷600=
0.19 min/day avg
11.4 seconds
11.4÷4=2.85
11.4÷3=3.8
from fsrs4anki.
Simulator needs a switch to disable Anki simulation. Optimizer needs to support a new card limit.
Please open a new issue to submit feature request.
from fsrs4anki.
Related Issues (20)
- [Question] Date change HOT 6
- [Question] Does fsrs deals well with cramming HOT 1
- [Question] HOT 21
- Optimiser Stops Working After a Single Review HOT 23
- [Question]: Questions regarding Optimisation HOT 3
- [Question] Should I use a low value produced by optimal retention (75%)? HOT 6
- [Question] Can FSRS schedule intervals shorter than a day? HOT 8
- [Question] What is the correct way to determine the card state (New, Learning, Review, Relearning) when implementing the algorythm separately from Anki? HOT 2
- [BUG]: Intervals don't update after changing Desired Retention HOT 12
- [BUG] "Ignore reviews before" Not Working HOT 31
- [Feature Request] Uneasy day
- [Feature Request] Decreasing load balance
- [Question] Wiki—post-lapse stability example parameter inconsistency? HOT 1
- [Question] Do the stability and difficulty not change when the card is in `Learning` state? HOT 1
- [Question] HOT 3
- [Question] Confusion about short term scheduling HOT 10
- Improve Handling of 'Set due date' HOT 45
- [Question] HOT 5
- [Question] Very long initial intervals when using multiple card types HOT 3
- [Other] Let's use the FSRS Simulator to figure out how to implement Load Balancer HOT 59
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fsrs4anki.