I see no problem with house ruling that a mercenary or other combat-oriented Hireling gains 1/2 XP like a Cohort. Oh wait, that does open a can of worms. A PC Cleric gains XP for the encounter if all he does is help heal the Fighter right? So what about a Hireling apothecary that uses the Heal skill to stabilize wounded characters in the middle of combat, shouldn't he get at least a 1/2 share of the encounter too? What about the Hireling scout that uses his Wilderness Lore skill to help the party avoid a quicksand trap? Etc, etc.
Re: "I guess it boils down this this. What is the difference between a 1st level warrior who's a Cohort and one that's a Hireling??"
I use the info under Leadership on pg46 to help adjudicate this rules dilemna. Therein, the difference between Cohorts & Followers is described as being, amoungst other things, their class options. I think the difference between Cohorts & Followers parallels Cohorts & Hirelings very closely. Using the DMG Follower rule to guide you, you could imply that a Cohort can be of any class, while a Hireling is limited to the Commoner, Expert, or Warrior class. That doesn't really answer the heart of the question, but it does describe another potential difference.
To rationalize the DMG rule that keeps the mercenary Hireling from gaining any experience is more difficult. The main reason is probably just to keep the spotlight of the story focused on the rising PCs, and to allow them to keep some minor retainers around for flavor without having a major drag syphoning off a large fraction of experience from the core party.
You could rationalize that the Hireling 1st lvl Warrior is not a hero, and is destined to be a peon for the rest of his life no matter what he goes through, while a Cohort 1st lvl Warrior (which would be a rare class option) is more like an apprentice or friend bound for greater things if he survives to learn and grow from his experiences (and could conceivably advance to add a level of Fighter or Paladin). The Hireling 1st lvl Warrior merely follows orders, he's not learning from the results of any choices he made because he makes no decisions, he is a subservient employee.
To use a parallel, if a person went to school or work and was ordered (the same way a D&D Hireling operates) to do everything, such as, "Add these two numbers together, subtract this number, multiply these two numbers, add that number, now stop you are done." Would you be learning from this?
Another student or employee (given the same guidelines for behavior that a D&D Cohort has) might be asking questions, trying out examples, and thinking about his mistakes, pondering over what's going on here, etc. Therefore, he seems to be more capable of learning and growing.
I realize that's a stretch, but it helps rationalize the situation at least a little bit.