I look for something that is particularly different and compelling, and then hang from there. I like coming up with what's unique about the setting and crafting a story around that, which couldn't happen in a "standard" faux-european-medeival-wtih-magic world.
Let me give an example of a D&D campaign I've got "on-deck" to show what I mean that the setting leads the story and the story the setting.
Refuges to Nowhere
You have grown up in the Tower. The tower is vast and long, and the angles are all wrong. Square rooms with perfect corners have five sides, and those who travel too far up or down sometimes come back not right in the head. Windows open in the tower occasionally, looking out on what you are told are other planes. Some planes open on a regular cycle, and planting has been done to harvest when it comes back around. Others are less predicable.
None are good places to stay.
You've been told that your world was ending. Maybe it was overrun by undead. Or the sun was going out. An age of ice. Turned into a battleground of demons and angels. Every culture in the melting pot has a different story. That you came to the tower as a last refuge. That so has everyone else, generations and generations.
But, you've passed your initiation. To be allowed to scavenge out the windows for the good of those who live inside. Be quick, sometimes a window closes in days, rarely more than a few weeks at lost. There are stories of people being lost and staying there until a window opened to the same place again. But you don't actually know anyone living that survived Outside that long.
a
This starts are an urban faction political crossed with a "dungeon of the week" style mega-dungeon. Which isn't actually a dungeon, but rather a window with a limited time for exploration. Depending on player interest, windows can reopen to "delve" deeper into each themed world.
Back in the Tower, there's all of the various factions living there, with allegiance to one or many, open or secret, getting gifted items, running missions, etc.
But the "Act II" of the campaign will be when the characters start noticing themes and such int he worlds - and thinking maybe the Tower has something to do with all of them being overrun. Then we'll have explorations "up" and "down" the Tower, as well as waiting for specific Windows to open to get information from specific worlds. They will find out more about the Tower (including mroe about controlling it and the Windows) and the nature of the apocolypsi that have overrun all the worlds that Windows open to.
This campaign may or may not be done with a stable of 3 characters per player. It will make copious use of downtime (partially enforced by the schedule of Windows, plus "no poaching" rules when other groups get access to Windows), this will be even more used if there's a stable of characters.
---
So as you can see, I picked a story that couldn't be told in a normal place, and a place that lends itself to unique play. Often these come up bit-by-bit. Here's oen where each sentence was a revelation that build off the last. For example, I wanted a world where the constellations changed depending on what was happening. So they weren't stars, they were spirits. So there's a literal dome between the living and the dead. Okay, the sun is inside the dome, so make it like Greek myths and an actual object a god drives around. And steal fromt here that it goes underground at night and travels back in tunnels. So what's underground? It's not the land of the dead, that's on the otherside of the dome. Hey, maybe the dome is fuzzy - not solid, just a zone. So you get close (on the ground) and it's sorta in the land of the dead and foggy, with the rules changing from land-of-the-living to land-of-the-dead the deeper you go. And you can go, like Greek heroes would go to the land of the dead and bring back souls.
I don't have a story that can only be told there, yet, but I'd got a plenty of unique to shape it.