I'd go with the 4-incher. Its a good compromise in the 40 caliber.
Have you shot a 40 caliber pistol before? Its snappier than a 9mm and some newer shooters that are not accustomed to recoil might find it a deterrent to becoming proficient with a handgun. Its not bad, just not as easy as a 9mm. Since I came from shooting 45acp 1911s and a 357 revolver, I was not bothered by it, but it IS snappier than my 9mms.
I'd suggest going to a range that rents pistols and trying several calibers and configurations before buying. It migfht help you decide which is best.
Pistols are a very personal decision and it MUST fit your hand well for you to be proficient with the thing. Example: I cannot shoot Berettas or Glock 45s worth a damn, because the pistols simply do not fit my hand well - it shows up on target...bigtime. However, the XD-40, 1911-style 45s and the Hi-Power fit me very well. Its a personal thing and you do yourself no favors by plopping down good money on something that looks cool, but does not really fit your hand well.
I tried for years to shoot my 92F Beretta well and finally gave up and sold it for a Hi-Power. I really wish I had not wasted my time on the Beretta. It is a well made gun, but it simply does not fit me worth a damn and it showed at the range! What good is a defensive pistol, if it does not feel like an extension of your hand and you're not totally confident in your ability with said piece?
Take the time to test-grip several types before buying...in the long run...you'll be glad you did. You may not wind up buying an XD at all...thats ok...get what feels best to you.
-Brickboy240