Just wanna give this a +1. Words that would be homophones if it weren’t for pitch accent are brought up a lot in these kind of debates but there are a gigantic amount of true homophones in Japanese, mostly words made up of two kanji and are read using onyomi. These words can be disambiguated in speech by explaining what kanji are used although 99.9% of the time it is clear from context. Pitch accent isn’t used for the sake of disambiguation, at least not primarily.
Pitch accent helps show what role a word is playing in a sentence and to set word boundaries (compounded words will only have one accent rather than maintaining the original accented points of the words that have been compounded, etc). I could go on here about this point but won’t as it is not my area of expertise.
On the main topic that OP has brought up, I would like to suggest that it is important to differentiate between being able to hear pitch accent and being able to pronounce words with a good command of pitch accent.
In the case of hearing, it is probably useful to consider that there are levels. It isn’t black and white. It comes with time, as others have said. The hearing side is much easier to get a grasp of than the speaking side. When speaking there is generally too much going on to be monitoring the intonation of every word you say (and it likely is counter-productive to do so in most scenarios). You tend to have more control during listening practice. Just being aware of it is a good start as a beginner.
In the case of speaking, most learners probably are more hindered by other issues in their pronunciation. Wobbly pitch accent is a hallmark of foreign accents in Japanese however using stress accent whilst speaking Japanese is a far bigger issue for most learners (in terms of how foreign they sound). I tend to believe that most times people think they’ve been misunderstood due to pitch accent issues it is actually due to inserting stress into words were it doesn’t exist and, the partner issue that comes along with this, elongating vowel sounds when they shouldn’t. I have seen this happen in real life many times: “I said the correct word so why didn’t they understand me?”
Essentially, my advice on this topic is to consider what your goals actually are and consider what you actually enjoy studying and then let that guide you. People do have quite strong opinions on this particular topic though 