A DM playing with a bunch of unpredictable players does not have that luxury.
No, but they do have the luxury of NPC's. What, are your PC's the first badass heroes in Adventureville? Ever? If not, there should be at least one retired adventurer wizard who takes a dim view of their antics, martials some of his buds, and goes on one last hurrah. Your PC's aren't the only one with the ability to wreak havoc, and if they are, the order should fight back...(iwth more or less force depending upon what kind of game you like to run).
As far as XP for NPCs goes, obviously a replacement for the kill things and level up model needs to be found.
I would respectfully disagree.
I'd say the world's greatest blacksmith certainly needs to do more than bang metal all day, just like the world's greatest warrior needs to do more than stab bushes all day. Blacksmithing can involve as much adventure as being a knight in shining armor, sometimes. Perhaps duergar have captured your main Adamantium supplier? Perhaps you've heard rumors of a living metal in the jungle that you need to check out? Maybe you'd like to live among the dwarves for a time, and get caught up in their war for a king.
I mean, you're an EPIC BLACKSMITH! Certainly you're not just shoeing horses for a living! You're having adventures! Not nessecarily the same adventures as the wizard, but still...you're at least tagging along with a nice party of questors, and even doing *that* nets you XP.
Similarly, I think the "greatest *blanks* in the world" should be the PC's, or other adventurer types. I know if I sat around buidling up my Craft (weaponsmithing) skill every time I leveled only to have the DM tell me that Mooky McMook-Mook was crafting better knives in his sleep, and he's never stepped out of his forge....well, that's just incentive to *not* go adventuring....screw questing, if I can make magic items sitting on bupkiss, why bother fighting mind flayers for it?
If you're the best farmer in the world, you don't just have a lot of ranks in Profession (sheepbeater)...you've got skills from farming through that plague of undead when you had to defend your family, and hitpoints from that time the druids went to war with you, and ranks in Craft (wool underpants) for that time the demon king of ice lorded it over you for a few years.
In D&D, anyone who isn't a mook is having adventures all the time. If you want your NPC's to not be mooks, give them some adventure, make them insteresting, develop plots around them where they go with the PC's. You don't get to be the best scholar in the world just by reading a lot, y'know...