I could be sprayed by a skunk ghast, but as long as I'm behind this wall, nobody can smell me?
But I get what you're saying. Anyone making a Stealth check is trying to be unnoticeable as much as possible, whether becoming hidden, silent, or covering themselves in mud to hide the skunk ghast...