I would have to think about it, but if you remove concentration, then I would shorten the duration of short rest for martial characters, or remove the attack action on your turn for features like extra attack. Basically if casters don't have to worry about concentration then martial characters should be more free to whittle down hit points.