This happened to my friend Frodo. The campaign started when he found out a ring he had inherited actually belong to a BBEG and he had to flee home. First few sessions it was just four level-0 halflings. They came pretty close to TPK several times, and the DM got tired of saving them deus ex machina style, so when a new player joined the DM let him start with a higher level Ranger with some pretty silly ability scores. (I'm convinced he fudged the rolls.). Only later did a Wizard finally join the party, but along with 3 more fighters.
A party of nine was too much, and Frodo really did not like one of the new warriors...total spotlight hog...so they broke up into smaller groups and played different sessions.