Your PCs must be the best-behaved party in all of D&D in that case, as they have clearly never:
 - attacked an innkeep after being shortchanged
 - captured a street urchin picking pockets
 - started (or tried to stop) a town riot
 - been involved in (or, as happened in my game, been the quarry of) a torch-and-pitchfork brigade
 - participated in a bar brawl with the locals
 - needed to neutralize a town guard in order to sneak in
 - needed to neutralize some prison guards in order to reach someone and free him
 - been involved on either side as a local farm gets raided
 - etc.
All of these require at least some mechanical representation of the commoners involved, even if their only further contribution is to flee, die, or do nothing.
Lanefan