I think the problem here is, for a game, Stealth should be pretty easy. Meet X conditions, hide. While hidden, roll Stealth vs. passive perception. Why passive? Because really, once people know you're around, finding you probably a foregone conclusion; there are only so many possible places one can be concealed in most circumstances.
We want to make Stealth more nuanced, by having situational modifiers, but the harder Stealth becomes, the more likely players will just want to not want to be trained in the ability- certainly, in D&D, the sheer amount of enemies with special senses that make Stealth difficult or impossible makes using the skill already a high bar.
Especially since, in many circumstances, you can't creep around with a light source handy (and "Dimvision" is just going to have you blunder into traps even if you have it).
And that's not even getting into fairly simple countermeasures- I once ran an adventure where the players were dealing with kobolds who had tossed caltrops all over the place, slowing the Rogue's progress to a crawl as he had to slowly sweep them out of his way without making any noise.
And of course, in real life, people are not as alert as they really should be; a lot of people can be snuck up upon in broad daylight because they lack situational awareness (also, facing is a thing).
So, like a lot of things in D&D, there's this tendency to make things harder than they really should be, because it's important to make the players feel challenged. Contrast and compare Persuasion or Deception; skills con men routinely use in real life to bilk people out of large amounts of money, but in D&D, every NPC acts super suspicious and skeptical of just about anything PC's say!