Dogs' sense of smell is hard for us humans to really comprehend. They can detect and discriminate scents that humans can't even notice and ten times the potency. It would be quite easy for a dog to pick out a scent from an item and then follow the person's scent through a city. There are some differences between ground and air-based scent dogs, but the most important factors are:
- How long has it been since the scent was left?
- What has the weather been like since then?
- What kind of item are the dogs using to acquire the scent?
- What kind of handlers do the dogs have?
- Are the prey trying to hide their scent tracks?
1. Obviously, if the scent is a week old, it's pretty much impossible (years and years ago, the record for a bloodhound successfully tracking a fugitive by scent was something like 4 or 5 days; that record might have been beaten, but probably not by much at all).
2. If it's been raining continuously for 12 hours, that's probably just as bad as dry weather for 4 or 5 days (and in a city, there can be all kind of things that have effects similar to weather).
3. If the item the dogs are using as their basis for the scent is something like a pillow piece of clothing (something soft and porous), it will be much easier for them than if it is something like a wooden cane.
4. If there are experienced handlers, they can help the dogs re-locate the scent when they lose it say, crossing the street (assuming the waste runs down the center of the street like in many medieval cities). Experienced tracking dogs will eventually be able to pick up the scent on their own, but the handlers are smarter (one would hope), and can make an educated guess as to where to re-acquire the scent, thus saving time and keeping the trail fresh. This could be a Survival roll on the part of the handler. The number of rolls and what modifiers the handler gets to use are up to you.
5. If the people being hunted aren't expecting tracking dogs to be used, they're unlikely to think about things to throw off the dogs. They might change their clothing to avoid visual detection, but never consider taking a thorough bath.
As for the Rule of Cool, if the PCs are trying something new and creative, I would give them a reasonable chance of success if the scheme isn't too far-fetched.