The factory applied oils are intended more as preservative effect than for lubrication. While a properly assembled XD / XDm should normlly function directly out of the box, you would be doing yourself a favor to clean out all that sticky residue before use. (keeps it off your clothes and hands at the range). The factory product is real effective at catching dirt and powder debris. Don't forget the mags, both inside and out for smoothest feeding.
Personally, I would never shoot a new firearm until at least a cursory strip - clean - inspect - lube routine. The pistol was test fired at the factory, but who really knows how it was handled and treated between then and now. So if not for ease and reliability of use, at least from a safety aspect knowing all the parts are there it would seem a logical point. Takes all of what, a couple minutes? If you can not do this in only a couple minutes, that is all the more reason to become more familiar with the workings of your firearm. And avoids any frustration or embarrassment of a possible first round or first outing failure of your new toy in front of an audience.