I have zero experience with MEXT but had some luck with public call applications.
Just replying to the title question, it’s one of those things that are not going to be a deal breaker, neither in the positive nor in the negative.
I don’t know what’s your spot in life and how you’re organized, but if studying Japanese is something that you like and doesn’t take your time and/or energy away from becoming better at what you do and from getting better grades (two VERY different things), it surely is an easy way to make your resume look better.
I don’t know when you’ll have you do the Japanese and English exams, but if it’s in more than one year, I’d focus on long term improvements in order to be as most ready as possible for that test, below the limits stated above, and take your last JLPT as close as possible to the exam date (in my country, Italy, it would be December of the year before applying for MEXT).
From what I read from Reddit (so not very reliable lol), the priority is this:
Having minimum requirements > having a good understanding of either English or Japanese > Having a great research plan and motivation on why you want to specifically go to study in Japan (what the country has to offer that no other country, especially yours, can offer) and how can Japan benefit on choosing you > Having a great academic record in what you do >= Having research activity about what you want to do in Japan > Understanding Japanese specifically.
My personal advice is take it easy in order to not burnout and play long term, but be at least a very solid N4 (like @Snappie texted, minimum level to get a good score in the easy section of the MEXT Japanese test) before taking the MEXT test if it’s gonna be in more than one year. The rest can always be a good extra to put on CV if it doesn’t distract you on what you like to study.
Best of luck, hope to be your colleague soon 