Rope. Always carry a coil of rope. Even on an urban adventure, rope is invaluable for climbing up buildings, tying up prisoners, rigging traps, and all kinds of other improvised uses. In the immortal words of Sam Gamgee, "You'll want it, if you haven't got it."
A knife is also highly recommended. You may never use it in combat, but it's immensely useful in all manner of noncombat situations.
Do you have a way to see in the dark (darkvision, the Devil's Sight invocation, a light-producing cantrip)? If not, a light source is essential.
If you have proficiency with thieves' tools, it's a good idea to carry a set.
And, of course, if you don't already have an arcane focus, you should get one. Keeping track of minor components is a pain.
Those are the main things I would make sure to have. Also, consider paper, ink, and pen, so you can write letters and messages; and caltrops, which can be thrown in front of a door to create a nasty little surprise for anyone coming through.
