Someone said something in here that gave me an AHA! moment; rolling Stealth to hide is dumb.
You're behind a big tree in a forest and use the Hide action and no one's around; what are you rolling against? What's the flat check mean? Why can you stand behind the tree and keep rolling Hide checks until you get a nat 20?
Hide should be an action. Then, when you do something that could get you noticed, you roll Stealth against their passive perception. Moving? Make a stealth check. Taking the dash action? Make a stealth check at disadvantage. Moving from one pillar to the next in a brightly lit area with guards present? Make a stealth check at disadvantage, hoping no one looks your way while you step out from behind cover. It's the middle of a fight and you duck behind a wall and try to hide so your enemy doesn't know when you're going to pop up and shoot your crossbow? Make a stealth check when you try to attack from hiding to see if you go unnoticed.
Then, be judicious about what observers are doing. Guard is dosing or chatting with another guard? -5 on their passive Perception. Guard is standing motionless and staring in one direction like Scotland Yard? Regular Passive Perception from the front, -5 to the rear. Guard is vigilant and looking around in every direction? +5 to passive perception.
You should only make skill checks when you try to do something. Move Silently. Snipe. Sneak past an open window. Sneak up behind someone. Sure, you can practice hiding by yourself and see how quiet you can be, but it should be per action.