Agreed. You would also potentially face balance issues as some sub-classes have very front-loaded features. In other cases it would be thematically rather problematic - how many Pacts is that Warlock making!? That Paladin has gotten himself into quite a pickle with those three Vows he's sworn.