Aye, using the 3e leadership feat only your cohort was high enough level to matter in most fights, and at most you'd bring a half-dozen followers along as non-combat support (healing spells, cart drivers, general henching).
Most of the rest were either never recruited, or sent to maintain a business, base, or other power structure.
Most of the rest were either never recruited, or sent to maintain a business, base, or other power structure.