It will get better. Once you finish N5 it will start becoming a little easier.
But if you still struggle - I recommend resetting the progress to zero and covering N5 again. This way you will properly sort all of the knowledge in your brain and make a good foundation to proceed to N4 and later N3.
I myself reset different levels several times and it makes it so much easier to learn the second time. Basically think of the first time as a preview, and then second time as the real study where you really understand everything.
Edit:
Btw, for reference my averages are
N5 - 93%
N4 - 88%
N3 - 78%
N2 - 75%
N1 - 71%
They could have been higher, but I am of the opinion that it’s better to “fail” a sentence on purpose if I want to study it (or the grammar point as a whole) a bit more. I’m not trying to win a competition here, but learn