The OP can speak for himself (and apparently has spoken quite a bit), but, actually, gangs, viking raiding parties, pirate ships, bandit camps, and wolf-packs being local anarcho-democratic autocracies makes plenty of sense to me. Sure, there's a leader who probably rules with an iron fist, but he was probably put in his place by the popular acclaim of the people and can be removed from his place by the same means. If slain, his particular qualities will be missing (and if he's a man like Black Bart Roberts, then the pirate fleet won't be what it was without him), but the gang, raiding party, pirate ship, bandit camp, or wolf-pack will go on without him. And, generally, said local autocrat doesn't owe his allegiance to anyone or anything.
Now maybe that's not the same thing banning BBEGs from your adventures, but a campaign can do just fine without them and will sometimes even be well served without one.
That said, I think that long-term campaigns lend themselves toward the eventual discovery of a BBEG for one reason: long-term campaigns tend to turn into high level campaigns. And when characters are high level, they are considerably more self-directed than at low-mid levels. An army of orcs on the march? No need to flee the city; we'll take them on by ourselves. 1000 to 1? So, you're saying it's a fair fight

. Was the character's life long goal to build a tower and create a school for young wizards? When you're 15th level, that's fairly easily accomplished. So, high level characters in a consistent campaign world can accomplish a lot of their goals and no longer need to adventure in order to gain the experience, wealth and fame necessary to do so. Furthermore, high level characters need each other less than low-level characters when it comes to dealing with the general challenges of the world. If your fifth level paladin wants to stop that company of 50 orcs, he's going to need help. If the rest of the party isn't on board, he won't succeed. On the other hand, when a 15th level paladin wants to stop a company of 50 orcs, it doesn't matter if the rogue would rather loot the crypt of the invincible overlord. Alhandra the 15th level paladin can defeat the orcs by herself. And, while 15th level characters can't do everything they want by themselves, they are much more capable of picking and choosing their companions through the use of their influence, feats, and wealth. Lidda might not be able to find someone as skilled as Alhandra the 15th level paladin to help her with the tomb of the invincible overlord, but she can probably use the guild connections to find someone who's good enough to meet the challenges that she needs to face.
So, what keeps the party of high level characters together and on track? Shared goals. Loyalty to each other (generally earned at lower levels). Shared enemies. That last one is the big one. If the forces of the BBEG are mustering in the east, that's probably important enough for Lidda to put her designs on the invincible overlord's tomb on hold for a while. It's also likely to be enough for Alhandra to put her crusade against the wild orcs of the west on hold too. (Orcs will loot a few villages; the BBEG will enslave entire kingdoms). A BBEG provides a shared goal that is challenging enough that a high level party actually needs each other and important enough for them to put personal goals aside for a while. If you ask yourself what kind of thing would create a shared goal for a group of high level characters and would also be sufficiently challenging that they could not take it one individually (or, to put it another way, is sufficiently challenging to provide the players with interesting and challenging encounters), it starts to take on a lot of the characteristics of a BBEG.
Whizbang Dustyboots said:
But gangs, raiding parties and wolf packs being anarcho-democratic communes without a leader, that would make more sense to you?