TwoSix
Everyone's literal second-favorite poster
If I wanted to do that (and I have thought about it), I would start with 2 poles of class design.Also I am not a big fan of having the system balance everything around combat. I want some characters who are good at non-combat things but not great in combat
You have a warrior class, which gains hit points, attack bonuses, and the ability to use magic weapons and armor.
You have a mage class, that does NOT gain hit points or attack bonuses, but does gain spells, and the ability to use scrolls, wands, and utility items. The mage is mostly worthless in combat, all of their magic generally takes at least a minute to cast.
You can differentiate and create more class archetypes within those two groups, but you do NOT hybridize or cross the streams. Warriors solve fights, mages solve non-fights.