Perhaps instead of a fixed HP per level, it could be rolling Con mod number of hit dice, then choosing the highest 1. Edit: specifically when adding more HP upon level up.
In that way, there is still randomness involved, and Con mod has an impact, but HP bloat is reduced, and Con mod just adds...