If your table is all 3E/PF savvy, it'll be fine, and everything I say is not relevant because those types of players enjoy the system mastery mini-game, but I know it would make skills opaque and inaccessible to all the new players at my current table. But, I have a critique if you're open to it.
--
I'm going to poke at this, if you don't mind, because I think Bounded accuracy basically accomplishes that now, and the mechanic you're proposing won't add enough to the game to merit the complexity.
Bounded accuracy means DC range is relatively narrow for most things. The idea isn't to have a constantly escalating DC over time like some previous editions, but to keep using 10-20 for 90% of all checks.
With that in mind, most players are in the middle ground. That's what 'Proficiency' is. Rogue/Bard expertise is the 'High' ground, and unskilled with only natural talent (or lack thereof) is the 'low' ground.
The fact that 'Proficient' characters all move along in relatively the same pace means the system can make assumptions about DCs, and the odds of players making or failing checks, and build monster/obstacle math around that. If the skill system is completely 'free form', then players have to develop system mastery to even know what a 'middle ground' is without wasting points accidentally.
So, I'm sure the right players would make it work and enjoy the granular tweaking, but it's definitely not for everyone.