LipSurf - Bunpro Voice Dictation. Review while you eat, do chores, etc

Bunpro
#1

@mikob, the creator of LipSurf, a customizable and fully extensible Chrome extension that lets you do anything with your voice, has announced that a plugin has been created for Bunpro. The plugin was created by @michiakig.

From @mikob:

What does this mean?

  • improve your recall by activating a different part of your brain
  • get comfortable speaking Japanese faster
  • use Bunpro hands-free

If you have feedback, we would love to see it on the LipSurf forum post (in step 2 below) so we don’t hijack this thread. See these two posts on how to get started right away:

  1. Install LipSurf:
    LipSurf - WaniKani Voice Dictation. Review while you eat, do chores, etc
  2. Install the community Bunpro plugin:
    :tada: New Bunpro and Kitsun.io plugins now available! - Official - LipSurf

Thank you @mikob for this amazing service and thank you @michiakig for creating the plugin!

Cheers!

13 Likes
#2

So excited to try this out when I’m at home! I’m already reading stuff out when I’m doing reviews, so this sounds exactly like what I need to make use of that. And since it’s available for WK too, hopefully my speaking can get a little levelling up from now on :partying_face:

4 Likes
#3

Oh wow! I’m definitely gonna give this a go!

4 Likes
#4

@michiakig, I have a suggestion! :upside_down_face:

I often read the review sentences out loud, so maybe LipSurf could be configured to only input your answer upon hearing a certain keyphrase? Like have it trigger when it detects me saying 答えは, for example. “The answer is… ______.”

6 Likes
#5

Thanks for the shout out @Pushindawood ! I’m glad there’s some interest and I hope the plugin is useful for folks. I’m very new to using Bunpro, so have not seen very much of the content and it is possible there are some issues, if anyone finds anything please don’t hesitate to contact me and I’ll do my best to fix :pray:

By the way I’ll take this opportunity to highlight how the plug in extracts answers from each card:

Array.from(document.querySelectorAll('.examples .japanese-example-sentence strong'))
        .map((x) => { return x.innerHTML });

So it depends on the example sentences, and obviously depends on the specific CSS classes which could potentially be subject to change. If there’s a more stable way to extract the answer let me know and I will update it (edit: actually of course it does depend on a few other DOM elements to submit the answer and click next and so on…)

Thanks for the suggestion. if I understand you correctly you’re suggesting to input the answer after the user reads the entire example sentence including filling in the answer? I believe that should be possible, although there are cases where the voice recognition API returns kana only or the wrong kanji due to homophones. I will experiment a little bit

3 Likes
#6

I’m having a hard time using this for BunPro. I went through the tutorials and I think it’s a really nice tool!

I just have no idea how to actually fill in the answer field at all. When I say “answer ____” nothing happens. When I switch to Japanese mode, I’m having a hard time having it recognize “BunPro” in “BunProの答え.” Once I’m in the review page, what do I have to say to make it answer? (I’m sure that the answer field is selected.)

Even when I manually write out the answer and say “answer,” nothing seems to happen

3 Likes
#7

Hi, thanks for trying the plugin, sorry you’re having trouble!

First, make sure that the plugin is active and loaded in LipSurf correctly. because Bunpro is a very dynamic site, in my testing sometimes the plugin didn’t load correctly and I needed to refresh the page (the plugin takes this into account and should “just work”, but let’s rule it out). Make sure that you see these two things in your browser, LS icon in Japanese mode, and Bunpro Context help (note you can hide the LS help)

BunPro%20LipSurf%20crop
BunPro LipSurf crop.jpg374×594 45.1 KB

If these are both there the plugin should be working. Then, you should say out loud just the prompt in blue, in Japanese. For instance, in a review I just tested the prompt was “is not”, so I said “じゃない”. The plug in should automatically submit this answer and press enter, if it’s working it will turn green as if you had typed it. Note that the plug in will only submit the answer if it thinks it’s correct, so if it mishears you or fails to find the answer for some reason, it won’t do anything.

After you say the answer and see it turn green, you can say “つぎ” to advance to the next card. You can also say one of these:

ひんと - toggle between hints
だめ - mark card wrong
じょうほう - expand the grammar info section

Finally, note also that because the speech recognition API is better with full context, the particle “は” needs to be pronounced “ha” and not “wa”, and “です” needs to be pronounced “desu” (two syllables). Unfortunately that is a limitation of the API (from Google) that I think LipSurf has no control over.

Please let me know if it is still not working for you

1 Like
#8

text input seems to be a premium feature

1 Like
#9

Will try again when my next review becomes available. But I’m sure I saw the BunPro help context there, and I tried it both in English and Japanese. However, I was wondering if maybe it’s because text input is a premium feature, as @mrthuvi mentioned? Or should the plugin work regardless?

EDIT: I added a new point and it’s working now! I don’t know what was the deal before, but thank you for helping me!

EDIT again: So actually, it seems to be working when I give an expected wrong answer. Like when I say わりに instead of のわりに. But even though it’s recognizing のわりに when I say it, it’s not going through. I’ll edit again with a screenshot.

image
image.png1919×724 77.5 KB

The voice input recognizes what I’m saying, but it won’t input anything in this case. I know it’s the right answer because I manually typed in the answer:
image
image.png1892×679 85.4 KB

4 Likes
#10

Great I’m glad to hear it’s working at least partially.

Just to clarify for anyone else, the Bunpro plugin will only expect Japanese speech (and only work when LS is in Japanese mode). I don’t know if there are Bunpro items that expect English input, are there any?

Thanks for the detailed bug report, this appears to be a type of grammar point that I haven’t seen before. I was able to reproduce the issue and will work on a fix.

3 Likes
#11

Awesome, thanks so much!

And you’re right to make the plugin only work in Japanese; no answers in BunPro are in English

4 Likes
#12

Here’s a new version of the Bunpro plug in:
https://raw.githubusercontent.com/okonomichiyaki/plugins/bunpro-new-timeline/dist/Bunpro.0-0-2.0.ls
To install this new version, simply click the link, use “save as” to download the file, then follow the same instructions to install a custom plugin for the first time

This should fix the issue with のわりに, along with a couple of other issues:

  • fix: activate on correct URL
  • fix: match answers that include particles, e.g. のわりに. this should work in general for answers with particles like this, however this particular phrase snags on the speech recognition in my experience (it hears 終わりに)
  • fix: match answers that include kanji. eg 全然, 来ます
  • improvement: fuzzy match on は particle to allow it to be pronounced “wa”

Please let me know if anyone tries this and has any further issues!

3 Likes
#13

Thanks to the Bunpro developer(s) for making an adjustment to the site code to support the plugin!! The new version is a vast improvement, in my limited testing. Answers with kanji, particles, conjugated verbs, and more are all working for me. Here is the new code: https://raw.githubusercontent.com/okonomichiyaki/plugins/bunpro_v0.0.3/dist/Bunpro.0-0-3.0.ls

Please let me know if anyone finds any issues with it

1 Like
#14

I’m locked from posting new replies due to posting 3x in a row, so will abuse the edit function once to post this update. Generally in the future, the latest version of the LipSurf plugin for Bunpro can be found in the LS forum thread, so if there are problems in the future, check there in case it’s reported.

There was a bug where some answers were matched but not submitted. This latest update should fix it. This also includes a new option to automatically show the answer if you mark a card wrong. Code here: https://raw.githubusercontent.com/okonomichiyaki/plugins/0c039069d505e1f7fbaf401f458b49213d251f8c/dist/Bunpro.0-0-5.0.ls

old

The latest version of the LipSurf extension was a major upgrade and necessitated some changes to plugins. Anyone using the latest LipSurf from the Chrome extension store will need to download a new Bunpro plugin. Here is the new code: https://raw.githubusercontent.com/okonomichiyaki/plugins/bunpro_v0.0.4/dist/Bunpro.0-0-4.0.ls

1 Like