IRT isn’t about comparing test takers with each other. IRT is about maintaining a difficulty level across multiple tests over many years. For this, the questions need to be weighed, and the difficulty values need come from something, so there’s some amount of comparing going on, but not necessarily between people who take the test together. The idea is that you get a passing grade if you’re statistically likely to have the required ability to pass past and future tests at the same level too. So it doesn’t really have much to do with how other test takers do in the same test.
Unfortunately, IRT makes it really difficult to evaluate mock test results. It’s not just that more difficult questions are worth more points. That would happen without IRT too. But for the statistics to work, test results can’t be too close to the extremes, or in other words, if some passes with 100% correct, you don’t actually know where they really stand because they could be at any point outside the scale too. So in order to avoid too many results getting too close to 0% or 100%, there are going to be questions that are way too difficult, but there also are questions that are way too easy. It’s possible to get a significant number of questions right and still score really badly because those easy questions will not be worth a lot of points. Conversely, it’s possible to miss some questions and still get perfect marks.
As for failure rates, I think part of the reason for the high failure rate is just that some people literally need the JLPT for something and will take it at any opportunity they can get. Apart from people living in Japan and working for a Japanese company, a JLPT pass can also significantly improve your CV in a number of other (mostly Asian) countries. But It’s not that we’d expect a pass rate near 100% for other language tests either, because those reasons apply to other tests too. If you think the failure rate is way too high, that’s probably just because you’re trying to be well prepared and trying to take a test that’s actually at your level. But not everybody does that. They might also count the no-shows as failures, too.
I’ve only taken the JLPT once myself, and only N1, but I also went through multiple years of mock tests on various levels. One thing that surprised me when I realised it was how few of the questions actually require understanding the text as a whole. Often times a question will only cover a small part of the text, in particular sometimes the questions for a text will just sequentially cover information in individual passages of the text. Another thing that often helps is underlining key words in answer options, and then underlining them in the main text too - if an option has a key word that just doesn’t appear, that option is much less likely to be correct. And when a key word does appear, the information you need to find out if the text agrees with the key word or doesn’t is usually really close by. Of course there will be cases where it’s not that simple, but anyway, if the text as whole is too confusing, try to see if you can solve the questions without that global understanding. And even if not, if you can just rule out one option, that already improves your chances to guess the correct answer. Don’t give up too early.
