FSRS for Bunpro Please

Yes you can (android here but iphone should be the same presumably)

1 Like

Commenting here to support the idea of improving the current SRS, but I think there are also other alternatives that could be explored by Bunpro other than FSRS. FSRS is great, way better than the SM-2 algorithm that Anki used previously, and people noticed a huge improvement. But that doesn’t mean it’s the best nowadays.

For reference, Anki is currently using FSRS-6. But here is the benchmark maintained by the same people who develop fsrs-rs, the implementation that is currently being used by Anki. And this is how the different algorithms compare against each other:

As you can see, there are WAY better things than FSRS-6. The main reason why Anki is not using them is because the top ones are DNN based (aka AI). The Anki maintainers don’t want to add something like RWKV-P because it needs to do inference on every single card, something that would consume too much power if running in something like a smartphone, which is understandable. It was discussed here, and just last month they achieved reducing the model to a size below that of FSRS-6 and implement it on an Anki fork for anyone to try on PC. It mentions only FSRS-7, but RWKV-P is also there as an option.

Although it makes sense for Anki to not to adopt this as they’re local first, since Bunpro is a web service first, it would make more sense having something alike here than in Anki, and it would put Bunpro way ahead of other alternatives in the SRS field.

Of course I’m aware of the difficulties something like this would entail, like how expensive would it be to run (it’s not something huge like running an LLM, but still…), or how it would mess up with offline reviews in the app. But I just wanted to share this to be taken in consideration for future improvement, and in any case, just like others, I’ll be happy enough with FSRS when it’s added :slightly_smiling_face:

Thanks again!

14 Likes

I just switched two days ago and am working through a newly added backlog of 2400 cards :frowning:

ahhh I didnt apply changes to old cards haha so its been very chill

2 Likes

Everything you guys say sounds like wizardry to me but it looks like everybody is pretty pleased with this FSRS thing so now I want it too.

9 Likes

I didn’t know about this fork! I’ve been trying it and it works well. Though I am a little confused if you are supposed to have both FSRS and RWKV active in the settings.

Sounds like hell. When you optimize the parameters in the deck options, you can leave the “Reschedule cards on change” off, so it’s only applied to new reviews. If you do reschedule and it gives you many (thousands in your case) dues right now, you can undo this (Edit → Undo or Ctrl+Z). You can also temporarily reduce the target retention rate and reschedule again, so the influx of dues isn’t so sudden. Usually no more than 90% retention rate is recommended (I personally use 85%).

Sorry, I missed this message!

Having individual toggles for both algorithms instead of a picker is intentional. The RWKV implementation there is experimental and not meant to replace FSRS-7, but to layer on top of it hijacking the scheduling of your deck. FSRS is still there doing its thing, computing intervals for your cards, and then RWKV overrides scheduling. That way you can see stats to compare yourself in what do these models disagree, which is of how devs are proof testing how well RWKV works over FSRS. They noticed a huge increase in cards memorized per day, but they also say it’s too early to attribute this to the algorithm change, since the period of testing has been for only one month so far, plus having a small sample pool. So we’ll have to wait, or jump in to keep testing and taking advantage of it in the long run :slightly_smiling_face:

You can see some discussion and results about it in the #fsrs-discussion channel of the Anki Discord. And you can follow the progress in the Releases of the fork. And just to note it, this is specifically mentioned:

RWKV’s model state and queue scores remain local to the desktop computer and are not synchronized. The collection itself remains compatible with mobile FSRS scheduling.

So feel free to test it without fear of messing up your existing FSRS decks (keep a backup just in case tho).

That makes sense. I’ve been testing this anki fork and sticking to FSRS 7 for now. I managed to migrate my collection from bunpro to anki now and will continue with that for the foreseeable future.

Will definitely keep an eye out on RWKV progress.

This is super interesting thanks for sharing! I haven’t used Anki in a long time, but I plan to get back into it eventually.

IIRC Anki supports plugins… Any idea if they support custom plugins that allow you to customise the SRS method?

1 Like

They mentioned in the linked discussion the possibility of implementing these as plugins, yeah. But to be fair, the workload for implementing something like this is not trivial. Definitely not a task for an end user, and something not to consider unless you want to start such a development project.

Plugins are also available only for desktop. The main mobile clients (official Anki for iOS and semi-official AnkiDroid for Android) have FSRS as common ground with desktop. With the fork that I linked, there’s currently little interest in adding SRS algorithms as plugins. You already have access to the newest iteration of FSRS plus the state of the art RWKV, integrated directly into Anki core, and it respects your mobile phone collection. There are also plenty of settings there for personalization, with FSRS-7 ones replacing the FSRS-6 in official Anki, and these added for RWKV:

I also think the SRS algorithm is so integral to Anki that it’s better for it to be tightly tied to the Anki client. You wouldn’t want an update of the client breaking your SRS due to version incompatibility. The lead maintainer of Anki has also expressed his disinterest in maintaining official plugins himself, so this fork might be the best option right now unless official Anki adopts these changes itself.

I think one of the problems with SRS (or FSRS) is that it is fundementally architected around Failed / Hard / Good / Easy button responses. To work with something like BunPro you would have to either rework the algorithm or map the users reponse to 4 possible values rather than the 2 at the moment (Pass/Fail). Something like

  • Failed: Got the answer completely wrong
  • Hard: Typo, or took >20 seconds to answer
  • Good: Got it right in <20 seconds
  • Easy: Got it right in <5 seconds

Although if BunPro has multiple sentences for each grammar point, you will also need to do something like keep the FSRS interval for each sentence separately and then use the mean or geometric mean of the scores for each sentence as the score for the grammar point.

FSRS works fine with only two grades, and I believe the majority uses it with the Pass/Fail addon. I also remember reading that it works better with only grades (tho it shouldn’t) probably because people don’t reliably grade themselves.

Making this time-dependent would be a nightmare. It might take longer to just read the sentence and understand the nuance.

4 Likes

Yeah, I also use Pass/Fail on Anki and it works great. I’d actually recommend it to anyone who’s still undecided about it. After you use it for a while, Easy/Hard buttons will feel like unnecessary noise.

And no time-dependent grading please. You may have just switched your attention to something else briefly or went AFK in the middle of a review. There’s no reliable way to tell what was the real cause for the user taking a long time to answer, and I bet most users wouldn’t even want to use something like that.

4 Likes

I’m trying to understand that thread…it looks like FSRS-7 is almost as good as the most expensive neural net, but small enough to fit on a mobile phone? So it’s pretty much the end all be all?

But it looks like the desktop fork still has a lot of WIP and rough edges…

It sounds like I can run the desktop fork without breaking my mobile client, but what happens when I open Anki on iOS and do some reviews? Do I just see everything scheduled with the old FSRS algorithm? Can I sync from mobile to AnkiWeb then, without breaking anything?

FSRS-7 isn’t nearly as good as the network based ones. You can check the accuracy column in the tables of the benchmark that I shared:

Algorithm Accuracy
RWKV-P 0.8450±0.0017
FSRS-7 recency 0.7394±0.0019
FSRS-6 0.6831±0.0045
Ebisu v2 0.5947±0.0031

I’m including Ebisu v2 just as reference because that’s the one with the worst score in the benchmark. And that’s already better than the old default SM-2 in Anki, which is better than tha basic algorithm Bunpro is currently using.

About the WIP stuff in the fork, that’s mostly related to the RWKV implementation, which is highly experimental right now. FSRS-7 however is pretty much complete. The same author of this fork has opened the PR in the fsrs-rs repo to include this officially, and then it could be added to official Anki, but all of this is so slow due to different reasons, if happening at all.

Regarding the sync issue, you’re right, whatever you review on phone is scheduled with old FSRS-6. It doesn’t mean you’re breaking your collection or your ability to sync through AnkiWeb. At worst, you’ll be limited to the accuracy of that older version, but just by syncing on your PC every now and then, the automatic rescheduling will already make positive effects.
Another main contributor is planning an AnkiDroid fork for FSRS-7 too btw.

It’s been a few weeks since I migrated all my vocabulary progress from Bunpro (using it now just for grammar) by bulk exporting notes with Yomitan and I’m overall very happy with how it works now. Throw in the mix a nice note template like Kiku and example sentences included in the dictionary definitions and I’m not missing much on anything.

Of course I’d be happy to have everything centralized again in Bunpro if they finally add FSRS and a way to import an Anki collection (Jiten is able to do that through AnkiConnect and it’s great), and also export it. Migrating my progress was a pain, and I really appreciated tools like Jiten having as a core concept the user being the one owning their data.

2 Likes

Quick question, would you mind sending me a DM detailing how your migration went, and how did Bunpro’s SRS levels map out to FSRS in your case?

I’m currently working on FSRS for Bunpro and while I can’t give a timeline, we’re taking a very close look now.

3 Likes

Please make it happen man this would be a dreaaaaaaam

1 Like

In fact, anyone on this thread who has successfully exported their Bunpro progress into a tool that uses FSRS as a scheduler, please shoot me a DM. I want to know how the main tiers (Beginner, Adept, Seasoned, Expert, Master) wound up feeling like/mapping out on FSRS.

Right now what I’m trying to validate is wether or not we could map an item’s predicted stability in FSRS into a Bunpro SRS stage as a start, before further optimization. Think of it as a baseline.

So, for example, if an item is 4 hours away in Bunpro, map that out into roughly 4 hours worth of stability.

1 Like

@Daru

Sounds interesting, but isn’t per item differentiation the main strength of FSRS? Bunpro levels don’t do a good job of showing the difficulty of a word, just the stability. For example you can have two words on the same level in say Seasoned, but one might have reached that quickly while another might have taken 15 attempts and a bunch of ghosts.

I’ll been using Anki lately but not by exporting Bunpro or trying to match the level, so I might try that and see what I find.