DnD 5e as a whole is absolutely awful at dealing with those concepts which are too much for a subclass, but redundant as a main class. Swordmage, witch, shaman, psion, and numerous others all fit into this weird place there if you the to make them a subclass it's an unsatisfying mess, while if you try to make them a full class they heavily overlap with something which is already there.
I think I'd love a 5e book that ignored the overlap issue. Think of the new classes like PF did for archetypes (substitute out as many or fewer powers as you want) or hybrid classes (combine two classes) and then do something new and present them as new classes (like PF did for hybrid). And if the problem is multi-classing, then just say they can't multiclass with the classes they're too close to.