Adding a -5 to Passive Perception with disadvantage is RAW, per the basic rules (p60, v0.2). But disadvantage only applies to Perception checks that rely on sight (p65).
So if the observing creature cannot benefit from sound, either due to being deafened, the hiding creature being silenced, or...