That’s fair, and personally I’m not learning from an LLM, but this is the whole premise of the application. So this advice is basically not far from “stop developing this app” 
How would you advice to approach it instead? A fixed list of sentences and a couple of permutations of translations, akin to duolingo or bunpro but with free-form input? That’s a mountain of expensive human work for a fraction of coverage.
If the premise of the application is providing potentially wrong sentences to its users, I personally don’t think it should exist. That’s not to say an app can’t have any errors (there’s a bug report button in bunpro for a reason), but it’s not by design and it’s much more likely to be wrong if a computer hallucinates.
That said, if the idea is to have LLM producing sentences and their translations, here’s what I would do:
I believe this would greatly mitigate the dangers of having incorrect computer-generated content.
This is a great concept and project, very helpful, and a great way to put studies into practice. I’m glad I was able to create an account during beta testing.
I wouldn’t pay any attention to anyone who doesn’t complement their studies with several tools and resources, with anything that adds to their progress and motivation. That being said, your initiave is absolutely awesome and definitely worthwhile using. Thank you
That’s fair. I’ll try to formulate my own thinking about it though I’m also struggling with understanding how I feel about all this. Let’s say you are currently learning Japanese. Your roommate and best friend is a pretty good Japanese speaker as a second language, but not a native speaker (let’s say their Japanese is about the same level as my English). They definitely make mistakes and you can’t fully trust their advice, but they are available to you almost all the time. You also have much more reliable sources (e.g., a Japanese teacher on italki, or your Japanese coworker) but your access to them is much more limited (e.g. you have a lesson with them once a week, or you only see them at work and feel much less comfortable asking them for advice about the language).
Would you avoid speaking Japanese with your roommate because they might give you the wrong advice? Or will you prefer to speak Japanese with them during the day to practice, since they are fully available and know Japanese pretty well, but know that you shouldn’t rely on them exclusively?
I think for me my approach to using LLM with my language studies is like getting advice from this friend/roommate. I know I can’t fully trust them, but I can still get some practice and useful information from them as long as I take it into account.
The difference is that your roommate can tell you when they don’t know something, and you can see them struggling to come up with sentences, so you know when they’re not completely sure.
LLMs hallucinate as easily as they provide correct answers, and they don’t know the difference (LLMs don’t have the concept of “knowing”). Even worse, they are often confidently wrong and you will not find out unless you look it up or you already know the answer. Even if they are correct, they can be “convinced” that they are wrong and change their answer.
They are just statistical models. Duolingo was mentioned as an example earlier ITT. Well, many duolingo users are not satisfied exactly because of the issues I’m listing:
So this app has taught me my grasp on particles when writing actual full on sentences isn’t quite as strong as I had previously believed 
but that’s okay!
However, a nice improvement would be if the correction blurb thingy included some contrasting examples and explained what the actual difference in nuance becomes as a result of your incorrect/unnatural choices.
Of course, maybe my grasp on particles is already perfect and the LLM is just hallucinating incorrect or unnecessary edits in an effort to dilute my actual knowledge and make my Japanese worse. If media have taught me anything, it’s that the machine is always out to get me 
(I’m mostly joking…mostly…)
Thank you everyone who has given feedback and reported bugs here and through messages. It’s been really helpful!
I’ve done a few fixes the last few days based on reports from users:
There are still many things I want to fix or add, but I’ll be traveling abroad for a few weeks, so Sakugo development will be on the back burner until I’m back in early August (I’ll still fix critical bugs during that time).
I just opened 10 additional beta spots in case anyone else wants to try it out. If you’re already signed up, you can click the “Click here to see if spots have opened in the beta program” message on the homepage to join (or go to https://sakugo.app/join-beta). I’m really sorry to everyone still waiting — since the backend LLM calls cost me money, I can’t open it to everyone just yet.
That’s hilarious 
I like how it commented on how the English sentence is unusual without realizing it came up with that sentence itself…
(it makes sense because currently these are two different instances of the AI tutor - one that builds the sentence and one that gives the feedback, and there’s no connection between them, so the second one doesn’t know that it’s an AI that gave the original sentence. Perhaps I should tell this to it in the system prompt…)
I forget what the sentence was but the same thing happened to me on day 1 or 2. Got a somewhat unnatural sounding English sentence, translated it as best I could, and got a comment on how the English sentence sounded a bit unnatural xD
I’ve also run into when starting practice each day it never loads the first try showing that my quota will reset in less than a minute, needing a refresh to fix.
If the English is unnatural, it should probably not complain about the Japanese being unnatural in the same way lol. It might be useful to have some way to report weird corrections from the page itself.
Will it eventually also track grammar you’ve used correctly/incorrectly?
Aw man, this is something I’ve been looking for as someone who wants to translate for a living! Awesome concept. I can’t wait until you let in some more members! I made an account just in case. 
Oh, thank you for reporting! I’ll look into that and try to fix, hopefully next week when I’m back from my travels.
Thank you for the report! I see a lot of complaints about unnatural English sentences and I have some ideas to fix which I’ll try to implement next week!
Regarding grammar - I’ve originally (before releasing the beta) had the app track grammar as well, but it seemed to detract from the quality a little which is why I decided eventually to remove it (it seems that with these small models, if I add too much things to track, it’s hard for them to maintain the quality level). It worked well if I switched to a more powerful model, but then every call is significantly slower and more expensive (around 30-40 seconds per sentence evaluation which made it very annoying to use the app).
Anyway, I’m still trying to figure out how to bring back grammar. I have some ideas for how to bring it back but I’ll probably only get back to it after I finish a few other things that I think are a bit higher priority.
I’ve signed up for this as it sounds just like what I need. Hopefully there will be space for more Beta users soon 
