I use this paradigm:
Roughly equal level to PCs, at PC level 1-4. Dangerous at equal odds.
One, two or three levels behind PCs, at PC level 5-15. Not a challenge at equal odds, but dangerous when PCs are outnumbered.
Generally no city guards above level 15, unless they are some sort of fancy...