I've always felt that, although actual Vancian spellcasters are fairly rare, there's also a lot of magic out in the world that doesn't necessarily fit into the "I'm an X level Y so I can cast this list of spells" model that PCs use...
When the books say that a particular village may not have seen any magic for a couple generations, at least to my mind they're generally referring to the "Holy ****, that guy just threw a ball of fire!" PC-type spellcasting.
In most of my games, there are occasionally folks (say, maybe in every third or fourth village or so) who can use some sort of very minor magic, either through natural gift or learning, but it's generally not the sort of thing that a PC spellcaster would learn (although I suppose they could, and might).
PC magic tends to be the chant-some-words/get-instant-effect sort of thing, where petty npc magic is more of the ritual casting sort where you spend minutes or hours (or even days) performing the magic to get a lesser effect or an effect that lasts over a long amount of time. If a random npc has some sort of natural magical gift, it usually doesn't directly track to a particular spell, although it might replicate parts of it.
Examples I've used in my games:
A couple individual npcs of different races have had the forest gnome's ability to speak to small animals.
Most country folks will tell you they know someone or know someone who knows someone who has the magical knack of being able to use the weather prediction function of the Druidcraft cantrip to various degrees of effectiveness.
There's a seamstress/washerwoman in one next town known for her ability to get out impossible stains and repair torn clothing like new who hums ritualized lesser versions of the Mending cantrip and the cleaning effect from Prestidigitation while working.
There was one old mountain man wilderness guide who effectively had the ranger's natural explorer class feature and a magical knack for tracking things that largely replicated the Hunter's Mark spell.
Old Man Wilkins in the next village earns part of his living by putting a "good hex" on folks' crops during the spring in exchange for getting some of the produce in the fall. He can't cast the Plant Growth spell, because he's not a spellcaster, but he can perform a two-day ritual that emulates the second function of that spell to a lesser effect - instead of all crops within a half-mile radius producing twice as much, anything within the area of an average farmer's field will produce one and a half times as much. He also knows various rituals to lessen the chances of your animals getting sick, or to cure them when they do. But unless your character is a centaur who's come down with hoof-and-mouth disease, he won't be performing any healing for the party.
Mother Graves is an herbalist and healer who has proficiency with an herbalism kit and the Medicine skill. Although she can't cast Cure Wounds, she knows a few chants that will heal an extra hit point or two over the course of a whole day, or a sprain or a broken bone in half the time...
There's a blind seer three day's ride from here who supposedly made a pact to give up their sight for the gift of prophecy. While they do have a divination ritual they can perform at will, it doesn't really replicate the effects of any particular divination spell. They mostly advise people on when to plant crops or provide insight on difficult life choices.
However, their mystical sight allows them to replicate slightly lessened ritualized versions of Detect Magic and Identify once per day. And once per week, they can perform a divination that has actual game effects - when asked a specific question, they can use a generic locate effect to find a person, place or object, perform a sort of lesser version of Legend Lore to provide various facts, and even give the person asking the question a single automatic success on one roll related to the issue.
Actual "hedge wizards" or "witches" that can cast spells and or make healing potions are fairly rare, and I generally make each of them up as a monster stat block rather than using the PC rules, since they generally only know utility spells, tend to have a lot of cantrips and I often choose their spells from more than one class's list.