Oh yeah, race-specific hit dice, I remember that the d20 Farscape game had that, and classes added a bonus modifier, and your class determined control point die and the race giving a bonus modifier. IIRC, that basically lead Hynerians Aristocrats being bad at both and Delvian Priestesses being good at both. Explains what got Rygel so mean-spirited, I guess.
Aynway, you better prepared to give the low-hit-dice species something worth the loss and not just say for verisimilitude's sake, they are worse at combat.
It might actually be better and remove constitution as attribute an acknowledge that hit points and hit dice got nothing to do with meat. (In any place you used to use Constitution, use Strength instead, except hit points, who are no longer affected by any ability score, unless you perhaps take a special feat or your class decides which ability affects hit points.)