The fact that we've had Sorcerers use CON for spellcasting since we started playing 5E and it worked fine. When we removed the link between CON and hp, it still worked well and made sense of course.
Consider that by Max level the wizard is going to have more hp than your con using d4 hp sorcerer if he uses ASI for maxing int and then con (possibly with resilient con in there). That seems off for a con focused class. It makes it obvious you are trying to design against the grain of the system.
Cool. I mean I don't think it's a game breaking change. I just don't like seeing a caster class like sorcerer equaling the hp of a fighter or a rogue without some tradeoffs. It's more a conceptual issue. If one changed the whole hp system of 5e such that all hp was tied to class (or most all of it) then such a concept works better (but that gets into changing the system to accomodate a class vs creating a class within the confines of the system).
Luckily the con based sorcerer can completely ignore int & gleefully put a rolled 3 in there with basically no loss. It's rare for a wizard to have more than 12 or maybe 14 con.
I don't think I've ever played a wizard with less than 14 con. Many of mine start with 16. In any event many wizards hit 20 con by level 20 and would have more hp than your con based sorcerer with 20 con.
More or less what we did. In short, we removed hp from any ability per level. You get your highest ability modifier at level 1. If your highest modifier increases, so do your hp. It helps reduce hp bloat. FWIW, all creatures lose CON bonus hp as well, so it across the board.
I personally think CON should have it's HP bonus lessened so people take CON to actually be tough.
Is the party fighting 4 Solars? 4 ancient dragons? 4 Balors? 4 Pit fiends? 4 Liches? If the party is fighting 4(or more) of a CR 20+ encounter, they're probably all dead before the fight begins.
