Before we start, I would like to preface this by saying there is no right answer - this is a "world design" consideration, and of course different DMs will come to different conclusions. But it's still useful to talk about it.
5e has given us several means to generate "petty" casters:
1: Someone who is a very low level caster and has "peaked" (ie won't go up in level and is "stuck" at that low level)
2: Someone who has a bit of magic via another mechanism - racial magic, or the magic initiate feat (a great way to make a "hedge mage" or priest), or a number of other feats that grant magic via a number of mechanisms - a peasant could be fey-touched for example!
3: NPCs don't have to follow precise rules. If you want a local fisherman to have a seagull familiar and not bother to explain how, go for it!
4: Someone could have an "external" source of magic they use - perhaps a magical item, or a magical being that can use magic on their behalf.
So these aren't people who are slinging fireballs and summoning angels - they just have a bit of magic that is useful to them and/or their community.
Personally I think it would be a mistake to have everyone have a bit of magic, but having every village have a local petty caster can be pretty interesting flavor. Of course once heroes reach tier 2 these will fade into the background, but they could be important NPCs at low levels. Just with magical initiate you can craft a great number of casters, who's style/flavor is based upon the spell list and the spell taken: "the ice mage" - someone with prestidigitation (for cooling and food preservation), ray of frost and ice knife, who's a rival of the red mage - a wandering tinker with mending, a raven familiar and firebolt. In an urban setting, there could be dozens of these. The Red Mage might have found employ at a petty noble, who wanted to emulate the king and get his very own "personal wizard".
Lastly, I'll mention that when I started writing this post, I had forgotten about racial magic, which would really change the dynamics in a community where this race is prevalent, but also how they are perceived. If there is one mage per village but every (high) elf knows magic... elves are magical beings!
Anyway, thinking about these "lesser casters" and how prevalent they are can really inform your world design and how the common people react to magic. Personally I like the "1-2 per village" level, as it helps with my personal "the PCs are not special at first but will become special" taste -at low levels you can cast firebolt, cool I guess, but when you start flinging fireballs people notice.