First: cool idea. These are fun, and your players are in for a treat.
Okay: It's really not that different from running a game on a continent.
Picture a map of a continent. Say, Greyhawk. You've got your power centers such as cities, tribes, monster lairs. You've got your borders marking the edge of the projection of said power centers. You've got your ruins and castles and haunted forests and other adventure areas. You've got your trade routes. You've got your mountains, rivers, oceans, and other terrain.
Now picture a city map. Say, London. You've got your power centers like the various theaters, courts, docks, nobles' estates, factories, police stations, bars, garrisons, temples, and guilds. You've got your borders that mark the projection of said power centers, such as "the business district", "dockyards", "rich district", and "market". You've got your ruins and sewers and docks and graveyards and arenas and other adventure areas. You've got your roads and other trade routes. You've got your rivers, barricades, gates, reservoirs, parks, and other terrain.
So you see that creating this city is really no different from creating your world. You're just "zoomed in" from a satelite view to a city view.
The fun part is that as you develop parts of your city you have a chance of getting a good return on your work. Meaning, unlike the work you put into developing the nation of Foo, which the players might never return to, you can be sure that the players will have many chances to appreciate the work you put into your city's most prominant thieves guild.
Specifics below. Don't agonize over this stuff, just jot down one-sentence or even one-word ideas:
Spend some time figuring out the history of the city. Why was it originally built? Did it start as a military outpost? Frontier trading town? Shipyard? Noble estate? Waystation? Lumber town? Mining town?
Spend some time thinking about how the city developed. How/why did it get big?
Spend some time thinking about the current state of the city. Why does it still exist and thrive? What is the primary business? Other businesses? What's changing?
Spend some time thinking about the texture of the city. How does it look? Smell? Taste? What kind of architecture? (think East vs. West Berlin). What characterizes the various neighborhoods? Are there street signs? What language(s)? (think chinatown) Are there street lights? Who lights them? Or are they magic?
Spend some time thinking of the mechanics of the city. What are the laws? How does it get food and water for the citizenry? How does it pay for its city services and government? Is there a mail service? Criers? How do citizens get around? Is there mass transit? Taxis? Are horses allowed on all streets? Are the streets dirt, paved, cobbled, wooden boardwalks? How does the city dispose of waste? How does it protect itself? How does it import raw material? How does it process raw material into finished products? How does it export finished products? (or the opposite, for a frontier town).
Spend some time thinking about magic. Do the guards have magic? Are there magical conveniences like lights, waste disposal, alarms? Magic shops? Are there laws regarding magic? What about magic items? How do people perceive the practice of magic? Do they differentiate between clerical and arcane? Is magical healing available?
Spend some time thinking about landmarks. Monuments. Temples. Fountains. Universities. Anything a tourist would identify as a defining characteristic of the city. Washington DC has the Washington Monument and Lincoln. St. Louis has the arch. New York has the Statue of Liberty. Seattle has the Space Needle. Venice has canals. Your city should have at least one feature or monument that would look great on a post card.
Spend some time thinking of the citizenry. Racial mix? General alignment? Attitude? (think New York vs. LA). Neighborhoods? (harlem vs. 5th avenue) Perceptions of own city? Perceptions of rival city? Jobs? Wealth? Education? Religion? Opportunity? Is the city growing or shrinking?
Then, spend some time thinking of conflict and opportunities of adventure. Start jotting down ideas for power centers or key figures. Don't get too detailed: "Joe owns a dive bar in The Docks, and has ties to the Thieves Guild and Pirates" is good. But brief concepts like "Cool, dark bar that serves flambe drinks" is good too.
Spend some time thinking about employers for the PCs. I'm thinking private security outfits, the city Watch, the garrison, the church, and of course thief guilds. But you could also include tailors, bakers, and the like--maybe the PCs have normal jobs at day, and adventure at night.
Generate Monster Manual-style stat blocks for typical Police, Military Garrison, guild thieves (maybe a different entry for each guild), citizenry, clerics, bar patrons, workers, nobles, bodyguards, private security guards.
Develop a handful of NPCs (not stats and stuff, just a few sentences on identity and goals). Guys like the mayor, the chief of police, the head of a couple guilds, maybe a few bartenders.
REALLY develop a smaller handful of NPCs. Guys that the PCs are likely to meet in the first couple adventures.
That should get you started.
As for running a game, remember to provide lots of detail on the various neighborhoods. When you say "Okay, you wander down to Foggy Bottom to meet your contact" the players should picture fog, brick buildings, tree-lined avenues, and the big monuments across the River.
If you want ideas on this kind of thing, you should pick up a copy of the computer game "SimCity". Or play through the game "Thief" or "Thief 2". It'll help you visualize how a city is laid out, and how a fantasy medieval city might work.
Good luck!
-z