Ghost Reviews (Update! July 12, 2018)

[Pushindawood]

Update: July 12, 2018

Choose Ghost Review Aggressiveness

• When Ghost Reviews are set to ‘ON’ in settings, a Ghost Review will be created for a sentence if you miss it once in your reviews. Set Ghost Reviews to ‘Minimal’ to only add a sentence to Ghost Reviews if you miss the same sentence more than once.

Tweaks to Ghost Review SRS

• Ghost Review intervals are now 4 hours, 12 hours, 24 hours, and 48 hrs, meaning that you will see your Ghost sentence four times over the course of the next four days.

Ability to remove all Ghost Reviews

• You can now remove all Ghost Reviews from your review queue by clicking on Profile, Reset and selecting Ghost Reviews from the dropdown menu.

Ghost Reviews

• Ghost Reviews are reviews that you have gotten incorrect and come back to haunt you (leeches). If you get an item incorrect, that particular sentence will automatically be added to your review queue in addition to your normal reviews.

• Ghost Reviews utilize a separate SRS so that you can devote extra attention to a specific sentence/nuance while you continue to receive new sentences and progress in your normal reviews.

• You will see your Ghost Review sentence six times over the course of one month. The SRS intervals for Ghost Reviews are: 4 hours, 8 hours, 24 hours, 2 days, 4 days, 8 days, 2 weeks, 20 years. *Intervals will be shorter if you get the sentence incorrect.

• With the implementation of Ghost Reviews, you will no longer get stuck in a review loop where, if you miss a sentence, you go back and see old sentences again. Instead, whether you get a grammar point correct or not, you will be able to further advance through the sentences on Bunpro! *Grammar points will still drop an SRS interval if incorrect.

Let us know what you think or if you have any questions. Cheers!

[seanblue]

It’s certainly an interesting concept. I look forward to seeing it in practice.

---

Instead, whether you get a grammar point correct or not, you will be able to further advance through the sentences on Bunpro!

Will there be any degree of randomness in the order, or will you get them in the given order still? After you review sentence 12, will your next review be sentence 1 again? I’m slightly concerned about getting sentences that are too easy at the end of the SRS lifecycle, especially since Bunpro sometimes quizzes in an easier way for early sentences.

Will sentences in your Ghost Review queue still show up in your regular SRS review if it happens to be that sentence’s turn?

Do you have any concerns about people getting too many ghost reviews? It’s not uncommon for people to get new items wrong a few times at the beginning. This could result in two or three times the number of reviews for this grammar point over a couple week period. Maybe that’s the point, but I wonder if it could also become overwhelming. I certainly wouldn’t change anything now, until people have experienced this version for a few months. But if it does end up being too much, it may make sense to not add Ghost reviews for the first 2-3 wrong answers.

---

I also noticed that you put the Ghost reviews on the Stats page. It might be a good idea to give them their own dedicated page. And what will you do if someone gets two sentences wrong for the same grammar point?

---

EDIT: Another question. Do Ghost reviews show up in upcoming grammar on the profile page? When I expand the “Show Upcoming Grammar” section, the two I got wrong aren’t listed as showing up in a few hours. And they don’t seem to impact the “Next Hour” or “Next Day” numbers either.

---

EDIT 2: I just did the first two Ghost reviews, but the summary page for the Ghost reviews says no reviews were done. Looks like there’s still some kinks to work out.

[Kamikazen]

20 years??! Is that a misprint? After 2 weeks and getting it correct, it seems like a long time to review that item again…

[seanblue]

20 years is their version of “never”. This SRS interval is just for the special “Ghost” review for the failed sentence. You’ll still get your normal reviews.

[Anthropos888]

The concept is great, thanks for implementing this. Nevertheless, besides all the issues which seanblue mention, I got the following bugs:

• I got one sentence twice in one session
• the review counter went to -1
• no ghost reviews shown in the summary, but I did at least 3 or 4 ghost sentences

Another thing: is it possible to handle the time span between ghost sentences more flexible? Example: my next review for that sentence is in 4 hours. But I don’t do any reviews for 2 weeks and then after 2 weeks I get the sentence correct because I could still remember the grammar. So it doesn’t make sense to set the next review for that sentence to 8 hours, right?

[seanblue]

Another thing: is it possible to handle the time span between ghost sentences more flexible? Example: my next review for that sentence is in 4 hours. But I don’t do any reviews for 2 weeks and then after 2 weeks I get the sentence correct because I could still remember the grammar. So it doesn’t make sense to set the next review for that sentence to 8 hours, right?

That’s a more general SRS issue I think. If they address that, it should probably be in general, not just for ghost reviews.

[Jake]

Randomizing the reviews is still on our list to do. It will most likely be pushed with the next big update. Even with randomization, you will still potentially run into easier reviews at the end of the SRS cycle. The only real way to deal with this is to have more sentences and that is something we are looking to do in the future.

Ghost reviews will indeed show up irregardless of whether that grammar point is also in the review.

Ghost reviews, when they are effective should actually naturally prevent too many from occurring. If you do find yourself with too many, it could be because you are going too fast, or you need to review the grammar more before continuing. That being said, all of N5 is a total of 93 grammar points which means even if you double or triple that, it is still a manageable about unless you are doing all 93 in the span of a couple of days. Additionally, these reviews will not persist for more than about one month which should prevent users from becoming too overwhelmed in the long run.

We are still working on design/placement of ghost review data. Ideally we want to show you what sentence it was that you missed, not just the grammar point. How/where best to do that is still up in the air.

I have added ghost reviews to the count for upcoming/next hour/next day. I have also fixed the issue with them not showing up in the summary.

@Anthropos888 That is an interesting suggestion. We will give it some consideration and see if we can implement it.

[Rosemary]

Wow - I hate the ghost reviews. Is there any way to turn them off?

My problem is that I often get things wrong through not reading the prompt properly and getting something wrong that isn’t related to the grammar being studied. I don’t want to mark these right, but equally, I don’t want the same sentence appearing again and again, as once I recognise it, I will just put in the right answer without thinking about it at all.

I would much rather go down a few points and advance to the next sentence. I’m getting way to many reviews this way, and I worry it’s actually getting in the way of learning.

[Pushindawood]

@Rosemary Thank you for your feedback! Before we implement a way to toggle Ghost Reviews on and off, we want to make sure that you know about the Oops! button that appears to the left of the Submit button when you get an answer incorrect. The Oops! button allows you to re-enter your answer if you feel that you have been marked wrong incorrectly or if you misinterpreted the prompt, without giving away the answer or creating a Ghost Review. You can even use the delete/backspace key for Oops! so that you do not have to move your fingers away from the keyboard.

I often get things wrong through not reading the prompt properly and getting something wrong that isn’t related to the grammar being studied. I don’t want to mark these right

Would you mind providing a little insight into why you do not want these types of answers marked correct? The better we understand everyone’s study methods the better we can tailor Bunpro to everyone’s needs.

We realize that Ghost Reviews may not work the same for everyone and we certainly do not want to negatively impact anyone’s learning experience. Your feedback is greatly appreciated. Cheers!

[Rosemary]

@Pushindawood I do quite often use the Oops button, but mostly when I mistype an answer. I sometimes use it if I’ve not read the prompt saying that I should use the polite verb form (for example), but that sort of feels like cheating, as I should read the prompts properly.

The main way I go wrong is when I make an error not connected to the grammar being tested - so say I need to conjugate a verb to the negative and then add the specific grammar point. If I conjugate incorrectly, then I’m still wrong, but it’s my conjugations that need work, not the grammar being considered. So I wouldn’t want to use the Oops button here as I’m clearly wrong.

Having the sentence constantly reappear as a ghost review - well, I suppose that would force me to conjugate correctly, but I think what would happen is that I’d instantly recognise the start of the sentence and immediately remember the answer, so I wouldn’t even be thinking about it or learning anything - I’d get them all right in the short term, but not actually remember longer term. All it’s doing is pushing my number of reviews up. And I’m very prone to giving up on SRS-based systems when the number of reviews becomes unmanageable.

I realise the above says more about my weaknesses than the utility of ghost reviews, but I still don’t like them! Given the option, I’d turn them off, as I don’t think they add anything, for me

[Pushindawood]

@Rosemary Thank you for taking the time to provide a detailed explanation of your study methods and how you use Bunpro! Comments like yours are very insightful and help us greatly in improving everyone’s experience on the site. We will get to work on implementing the ability to toggle Ghost Reviews on and off and should have something available for you soon. Thank you for your patience. Cheers!

[seanblue]

Will you consider more general flexibility with Ghost reviews? So far I haven’t had a problem (I’ve only gotten a few reviews wrong), but I am also concerned that they will really blow up the number of reviews I have. With that said, I’m not sure I’d want to disable them entirely because I think they could be valuable. I’d kind of prefer them to drill in the sentence for a few days and that’s it, maybe making the SRS cycle 4 hours, 8 hours, 1 day, and 2 days only.

[Anthropos888]

Ghost reviews are just implemented since a few days but I already feel that they help to understand and memorize the grammar better. For me they are a huge benefit. Before, the time span between the same sentence after getting it wrong was just too long. I forgot wether I need to conjugate the grammar point with a な or の or whatever. Ghost reviews make me remember these conjugations.

But I also think the SRS has to be more flexible, similar to Anki. When I get an item correct after 10 days it makes no sense to set the next review to 8 hours. It should at least be another 10 days. Or in general: the time for the next review of a correct answered item should not be less than the time between the last and the current review. And that differs from user to user.

[Rosemary]

Well, I may get really used to them and come to love them, so it’s possibly not implementing a button just for me, but thanks for listening to my whingeing!

[Kumi]

Another thing: is it possible to handle the time span between ghost sentences more flexible? Example: my next review for that sentence is in 4 hours. But I don’t do any reviews for 2 weeks and then after 2 weeks I get the sentence correct because I could still remember the grammar. So it doesn’t make sense to set the next review for that sentence to 8 hours, right?

@Anthropos888 That is an interesting suggestion. We will give it some consideration and see if we can implement it.

I would like to say that I have my Anki decks set up this way and it works very well.

[Jake]

I am not familiar with this style of SRS. Say you wait two weeks and get it right again, do you jump from your streak of one to whatever streak would be around that length of a wait? Like from 1-6 and then continue on from 7?

[Kumi]

The thing is that I don’t have SRS levels with Anki; there’s just a multiplier based on how well you say you know the item. I’m not sure how that would work when you have specific intervals set up, but if you were to implement this, my expectation as a user would be this:

Say you have an item at SRS level 1. You go away for three weeks and don’t review. Then let’s say there’s a 2 week interval between level 5 and 6 (I don’t know Bunpro’s intervals so I’m making something up). If you get it right when you come back, then that counts as answering the level 6 review correctly, and your next would be level 7 after the normal interval between 6 and 7.

[Kirito]

+1 for ghost reviews off. There should be an option to put them in a separate queue.

I got so many reviews today. If I keep getting this many, it’s going to be unmanageable.

[Kirito]

This needs to be fixed immediately. I’m not sure if this is intended for people with only short review queues or high accuracy, but it makes my review queue unusable.

Did anyone actually take the time to calculate how many cards this is going to add? I have about 60 reviews a day with 65% or so correctness. With ghost cards, I will have 20 new cards added to my deck, so the next day, I will have 80 cards. Not only this, if I keep maintaining 65% correctness, the number of my cards in my deck will spiral out of control, quickly to many hundreds.

[seanblue]

While I agree that Ghost reviews can spiral out of control, have you considered that perhaps you’re doing too much at once?