There are no classes with a d4 hit die, so as phrased, the question is moot.
However, if you had a Cleric, Wizard, etc. with 20 Con, then you could have a situation where 2x[Con Mod] is greater than the hit die.
Personally? I think the feat should give you that minimum value. So you...