I recently thought more about this and after revisiting this thread a couple more times I think I may have some more answers from a “linguistic” sense as opposed to just a practical one. If anyone else finds the study of language itself to be a fun topic this may be interesting to you as well.
In Japanese, there are obviously 5 main vowels, あ い え う and お. Between these are two “high vowels”, い and う, two “low vowels”, あ and お, and the remaining “middle vowel”, え.
I say “high vowels” for い and う because of tongue position. When you say a flat い or う in Japanese, your tongue is high in the mouth, almost touching the roof.
These are also vowels you see in “unvoiced vowel” phenomena, where a high vowel sandwiched by 2 unvoiced consonants are also unvoiced. Think how the phrase: 失礼します is spoken as “shtsureisimas” as opposed to “shitsureishimasu”, the “i” in “shitsu” is sandwiched between the unvoiced “sh” and “ts” consonant sounds, thus becoming unvoiced. likewise for “masu”, the “u” is sandwiched between the unvoiced “s” and nothing, and since “nothing” is unvoiced, so is the “u”.
I say all of that to explain this. When initially going over the あい → え casual contraction phenomenon, I had wondered if it was restricted to just this pattern. As Sidgr correctly noticed, however, there was also a おい → え pattern noticed.
I tried to think if there were any more examples of this I’ve heard in casual speech, when I realized I’ve heard “違ぇ” (read ちげぇ) before, which is a あう → え pattern.
That is where it finally dawned on me.
What if it’s not a “あい → え”, “おい → え” or “あう → え” rule, but instead a “low vowel + high vowel → え” rule?
This would make sense as え as a vowel sound has your tongue in the middle of your mouth, as opposed to “lower vowels” like あ and お, so as a “casual contraction” え would mean you didn’t have to move your tongue around as much, increasing the speed at which someone could talk.
I don’t have any proof that this is the “exact” reason why the rule exists, but it makes so much sense to me that I can’t think of what other reason there might be as to why.
I might take this reply and make a whole new thread at some point, but I figured it still related to the original question as there was no real answer as to “why” just that “it happens sometimes”.