I like the 3.x idea of commoners-with-levels for above-average skill levels.
Levels are how competent/experienced a character is overall. A high-level commoner is highly competent with what he does, even if it's not adventuring-related. A 7th level commoner could be a legendary farmer but only as useful in a fight as a 1st or 2nd level PC, a 5th level adept could be a master alchemist & expert sage even if he was only as useful in combat as a 2nd or 3rd level Cleric or Druid.
The trouble is that the commoner is also competent at adventuring related stuff. A 7th level commoner is better in a fight than most low level PCs. He's tougher than a 1st level fighter.
Now, a 1st level Commoner could be a passable journeyman in a trade with 4 ranks, a +1 or maybe even +2 from ability score, and a Skill Focus feat (and maybe another boosting feat if Human). +8 or +9 to a skill is piddly to mighty adventurers, but it can still do the everyday tasks of a profession just "Taking 10". A 3rd level commoner could even get a synergy bonus, a couple more ranks, and another feat which might be in something related and easily get over +10 and be easily able to masterwork items and could fairly be called a "master" of their trade. There is little need for even mid-level NPC classes to fairly depict even the overwhelming majority of NPC's.
A PC can be just as good (and probably better) without even performing the skill for a living. This is the problem. It dulls the motivation to have ties in society.
The whole "0 level" concept never made sense to me, the idea that the only way to gain levels is in combat-related, adventuring-oriented styles of training (i.e. classes) and that a brand new PC who has never actually clashed swords with an orc or cast a spell in battle is more worldly (i.e. experienced/levelled) than a master alchemist or swordsmith who has been practicing for decades.
1) It's no different in 3e, since levelling, as a function of XP, is based on a system of adventure awards. This doubles the problem since not only do you need levels to replicate PC level skill, but granting NPCs those levels is totallu arbitrary since they wouldn't earn them in the first place.
2) It makes no sense for most NPCs to become more resistant to threats and better at combat because they've improved in professional skills. We let PCs do this because we assume that as adventurers they are tested on multiple fronts in extreme situations.
Secondary Skills, for a similar reason, also didn't make sense. If you didn't start with one (which were randomly rolled anyway), you couldn't learn it. You might have been an armorer or a fisherman or sailor before you began adventuring, but there was no learning any of that after that first dungeon crawl, even if you spend most of your later adventures after 1st level in a seaside fishing village based out of an old smithy.
The 1e DMG is very clear that Secondary Skills represent a life before adventuring and that adventuring is a rigorous enough pursuit to take up time that would otherwise be spent learning a trade skill. In 1e adventurers live in a world where most people are not heroes, but are still important, and skilled in ways adventurers are not.
Now in 4e you can have this just by designing most people as monsters, since 4e monsters need not obey the PC framework at all, but you still have the "Paragon Ice" issue. A flaw 1e and 4e share is a lack of support for bringing adventure-marginal skills (like various craft skills) into focus as meaningful character trait. 1e assumes you'll never learn them and 4e defines "skills" as things you would do in an adventure, period. For all folks complain about 4e being MMO-like, MMOs actually do this at times, and some adaptations (such as EQD20) worked with this.
It seems to be that in the end, each system has its flaws. Right now I'm busy with a skill system that is organically evolving from playing AD&D1e RAW, then gradually changing things. I've identified 4 categories of skills in the process:
1) Trade or professional skills unrelated to adventuring -- the Secondary Skills system.
2) Skills where being an adventurer does not put one at a special advantage over 0-level types. I use ability score checks for these.
3) Skills that relay on an adventurer's overall talent and experience, but are not race/class-dependent per se. I have folded these into saving throws.
4) Class/race dependent skills.
Now #4 has led to an interesting situation with regard to thief abilities, since many of them are things anybody should be able to attempt. I have decided to work with them by emphasizing their definitions. Anybody can sneak, but Hide in Shadows is equivalent to Invisibility. Anybody can try to climb, but Climb walls allows sheer surfaces without tools that non-thieves (or monks, etc.) can't attempt. I am also allowing these as rerolls for more mundane attempts at things anybody can do.