This seems like an awfully harsh requirement to place on yourself.
I would respectfully disagree with the idea that counters are strictly “grammar”, in Japanese specifically. (I believe you mentioned that you know other languages in another post, so I want to be considerate of that background.) Japanese words change meaning by some conjugation and some agglutination (adding on) of words/particles. It’s difficult to strictly separate Japanese words into “grammar” and “vocabulary”. At beginner levels, textbooks present Japanese words in that context because that’s what European language speakers are used to. But with more “advanced” grammar, you see things that blur the lines, like words that operate as nouns but have to be translated as verbs in English.
Literal: There is no thing that (I) went to Japan.
Natural: I haven’t been to Japan.
– The way to say “haven’t been”, which is wholly a verb in English, is translated using two verbs and a noun.
(fyi - I grew up speaking English, I’ve studied intensively Japanese and Spanish, and I’ve dabbled in German and Korean.)
Back to this thread’s topic, yes, Japanese counters are difficult, but there’s only a few that are weird (一人、二人、三人…), and you pick them up pretty quickly as you encounter them in conversation. The resources that I’ve seen have typically presented counters as vocabulary. On a practical level, there’s little consequence to not knowing all the counters: standard tests like JLPT only quiz you on a few of them, and when talking to Japanese people, they help you through it because they know it’s confusing. That said, probably adding a few counters to Bunpro in the future might be a worthwhile update.