Three items:
- First, let another well-experienced handgun shooter (preferably one who has a good track record with the XD/XDm) shoot the gun, both two-handed/freestyle as well as single handed (left handed, in-particular).
It's always best to first take yourself out of the troubleshooting tree if at all possible, no matter how experienced of a shooter you may be.
- Second question is whether if you have tried different ammo. Simplify the equation even further and try different, factory-new ammo.
Sometimes, one specific gun just won't like one specific ammo. No rhyme or reason (actually, it's just weird tolerance-stacking). Always vet your ammo, in case you haven't done so already.
A subset of this is whether you have bee shooting the same box - or even same lot - of ammo, and that's what's giving you problems. You're right, 4 in 90 is excessive, but in my experience (although I haven't been shooting all that long, I did go through the '12 craze, and shooters who lived through that period can attest to the Q/A and Q/C problems we encountered, even of highly-regarded factory new ammo), it's not unheard of for a single box/lot of problematic ammo to present with such high numbers.
- Next in line would be magazines. Have you tried more than one magazine. Assuming that you have your magazines uniquely identified so that you can absolutely and positively ID individual mags, can you trace stoppages to (a) specific mag(s)?