I can see Prestige classes as being a replacement for feats/ability improvements. Lets say you enter a PrC at level 8, from here on out when ever you would have gained a ability score improvement or feat, you instead gain an improvement from your PrC. This would lock you in though, so you can not go back later and collect more feats/ability score improvements that are not a part of your PrC unless your class gives you that option at different levels other than 4,8,12,etc like Fighter.
For instance: A wizard takes the PrC Archmage at level 12. At that point she/he can not take the normal ABI or feats, but she/he can select a enhancement from the list provided under Archmage like a one off Metamagic ability used once per short rest, an ability that allows you to convert any spell of one damage source to another, or an enhancement that gives you +2 int and one additional spell know, etc.
This method would keep your core class complete, be easily balanced against feats, be accessible at multiple spots in your adventuring career, yet be flavorful enough to become a dominating aspect of your character. Being able to enter once every four levels ( although very restrictive ) does allow the player and DM to work the PrC into the game prior to the character gaining the advancement which would allow for a more natural inclusion in most games.