If the location isn't specified in the module (assuming there is a module) ala CotSQ, the pragmatist in me says go with a city for several reasons.
1) A massive dungeon-crawl spanning several to many character levels is going to include a lot of treasure, much of which will be disposable (gold, gems, etc) or items the PCs don't want (i.e. the fourth +1 long sword). Having a city with a large population and subsequent wealth level makes it easier for the PCs to trade their unwanted treasure for things that benefit them, and the CR/EL vs APL system includes the notion that the PCs are appropriately geared for their level. If you have a small village, they can't do this without long-distance travel, though if the PCs are already high enough level to have teleportation magic, this becomes a smaller concern.
2) Along the same thought-process as #1, if the PCs are below 9th level they may need spell-casting services from an NPC that will be difficult to acquire in a village. Honestly, even if they are 9th level+, if the party cleric bites it and they can't get him rezzed by an NPC, it's reroll time.
3) I personally don't like a dungeon that doesn't make some modicum of sense (YMMV), so I tend to assume that the creatures that inhabit the dungeon are there for a reason which usually involves some plan ranging from mischievious to diabolical. There's probably little in a village to interest the army of monsters represented in a massive dungeon, while there's a much broader list of potential goals for the rulers of a massive dungeon in or around a large city. Admittedly, a lot of this depends on the specific dungeon. A massive fortress built on a magical node and ruled by a lich, populated by his undead servants and foul experimental creations could easily and explicably exist in a remote location, but as a general rule, I'd say they'd either be near a population center, or far removed from any humanoid presence, both of which leave little room for the village.
