I fail to understand how I can possibly be ignoring skills and proficiency just by calling for checks normally. This is a bridge too far -- that the game doesn't acknowledge or represent skill or proficiency unless a passive floor is installed. That's just a weird thing to say.
I don't allow passive checks to be a floor. You end up with things like a passive 25 perception check with a skill bonus of +10 (observant feat). This means that the PC autosucceeds on very hard perception checks at all times (absent disad) but can only succeed on an observation skill check actually rolled about 30% of the time. That's just odd stuff, there. To make that work, you'd have to have a host of other houserules to level it out, and I'm already past my limit of keeping track of PC stats if I tried to remember what their passive scores are. Way too much work for wonky results and not much improvement over just calling for checks when appropriate.
Passive scores represent constant effort over time. It's the score you use when on watch, or if you're looking for traps down a long hall, or other, constant effort tasks. Normally, according to the PHB play loop and using the middle path from the DMG, you'd only ever call for a check when the PC states an action with an uncertain result, the task is achievable and not trivial (this is based on the task, not the PC stats), and there's a consequence for failure. That pretty much solves the calling for trivial checks problem, because you'll only call for checks when these things apply. And, the neat thing, is you don't care what the PC stats are -- the DC should be based on the difficulty of the action attempted. So, if you call for a check and the PC can autobeat it with their stat, they feel super awesome and the thing happens. It's a neat way to do exactly what you're talking about -- making PC skills and proficiency count -- without ever even having to think about what the PC's stats actually are. They're going to try to attempt actions that align with their abilities and PC desires, and you just adjudicate and it comes out in the wash.
And, I say this after having tried what you're talking about. I even further codified skills and bonuses to attempt to achieve skill perfection. It took getting frustrated and then actually listening to a few other ideas from posters here (shocking, I know) to get to the point that I was doing way, way too much work to get a result I didn't actually like. I tried the rules, and, wouldn't you know, they actually work pretty well. I don't fret it, my players are having more fun, and I'm having more fun. It's cool.
But, that's my way, not THE way. I'm glad you have a system that works for you, I just find it to be way too much work for not enough payoff. Especially the knock-on effects of having to houserule other things to fit it in and the DC inflation.
It wasn't hard to address the DCs. With or without Passive scores as a floor, I disagreed with the published ones anyway. In addition, I didn't care for the huge range of bonuses once Expertise was factored in.
As for the game "ignoring skills and proficiencies" it has to do with assessing when a roll is needed in my opinion.
For example, common advice now is to not ask for a roll when there is no chance for failure. How can you determine that if you don't compare it to their skills? If you asked my daughter to play a G chord on a guitar, she'd have no way to do it. But it's trivial for me, I don't even have to think about it. Likewise, give me a chord chart and I can play along without any trouble. They are both within my capabilities in nearly any circumstance where you give me a guitar. That would be within my Passive score.
If you give me a guitar in a different tuning, then I'd have disadvantage, because I'd have to figure out the fingering. It would depend on how different it was from standard tuning, though. So I'd have disadvantage and would probably have to make a check. However, give me a little time and I'd figure it out. I could still reasonably fail in future attempts, though. So if I'm just sitting there working it out, no check is needed because I have all the time in the world. But put me on stage and I have to perform it now? I'd have to make a check because it's not within my Passive capabilities and I might fail.
So as a DM, would you require me to make a check if I was given a guitar in standard tuning and told to play a song for the king? If so, then I think that you are not considering my skill or taking it into account. I can play that without fail no matter how many times you have me try. Forcing a check indicates that I might fail.
So, how do you determine what's within my capability and doesn't need a check? You do it on the fly, perhaps. But you base it on something. The obvious thing to base it on is DC vs. Passive score. That way you'll know that my +7 in guitar is significantly higher than the kid I started teaching a year ago and only has a +2. And that there are a lot of things I can play without thinking about it, that would be a challenge for them.
Perhaps paradoxically, I'm not trying to get "skill perfection" and one of the things I prefer is a bit more randomness in the game. That is, instead of a +x bonus, I prefer a +1d4 bonus, for example. Where possible anyway. Because I'm not a fan of the way a codified ruleset strictly defines success and failure. Having said that, I do think that for any skill, there are levels of proficiency (and non-, proficient, and -expert are enough for me), and that by tying a variable (advantage) to a fixed floor is a good way to cover both of those design elements. Defining DCs better is what helps ensure the floor is neither too low or too high.
It makes the game much more streamlined, in a way where there is solid information for the DM to work with when running the game. The focus can be more on the situations and circumstances that alter the normal chances of success, as well as on requiring checks when the stakes are higher, and avoids asking for what often seem like trivial checks.
DMs make these decisions all the time. But tying them to the Passive scores means they can make them more easily, and also take into account each PCs skills more consistently. It's less work (other than any tweaks you want to make to DCs, if any).
To address your Perception example, that's already covered in Sage Advice. You don't not see the stuff you'd notice with the bad roll. You'd simply not notice anything else. In RAW Passive Perception is already considered a floor.